Skip to contentSkip to conversation

End-user impacts and change log

Proposal Joseph Reynolds 2 participants 2 messages Feb 26, 2019 – Feb 27, 2019

lore
Share & tools
lore
#0 lore raw

This is a proposal to improve OpenBMC communication and get better release notes.

** Preamble **

Teams that use OpenBMC in other projects will not know OpenBMC as well as the development community, and they will not know what is in each release. The main way for them to find out is by reading the release notes. They will make decisions based on its contents. That makes the release notes an important communication between the development community and its end-users.

We need to get better at communicating new and changed OpenBMC functions. Doing so gives many benefits, including better release notes [1].

** The proposal **

1. Adopt a new practice to provide end-user impact statements when and where it make sense to do so.
2. Begin a project level change log to summarize user impacts.

** Details **

How do these changes help connect the folks who contribute to OpenBMC with the folks who use it? The main points are:
1. Contributors write an "end-user impact" statement to notify users that something they might care about has changed.
2. Community members use end-user impact statements to create a change log which summarizes changes they consider significant.
3. Release planning summarizes the change log as an input to the release notes.

Items in detail:

1. Create end-user impact statements.

An "end-user impact" statement summarizes each change from the point of view of someone familiar with using the external interfaces. It tells them what they need to know about what changed from their point of view, so they know what to test, how to adapt, etc.

When there is an impact, it should have details like:
- Is there an end-user impact (y/n)?
- Is this a new feature or function (where are the docs)?
- Is this a bug fix?
- What does the end-user need to know (one liner)?
These statements don't have to be perfect. Any inkling you can provide that there is a user impact is helpful to the community. The impact might change during the release as the design evolves. Different contributors will see different impacts. That's all normal and okay.

Are the following end-user impacts?
- Change to phosphor-webui that affects the browser display? Answer "no" if it doesn't affect usage, or "Yes" if the user will likely notice or care.
- Bug fix in fan control? Answer depends. Answer "yes" if the bug might start a fire.
- Refactor code? No
- New Redfish API? Yes; new features are positive user impacts.
- Adding new documentation? No; although helpful, docs don't affect the end-user's use of the external interfaces
- BMCWeb retires old and accepts new TLS encryption algorithms? Yes, it affects how you can connect to the BMC
- Changes to phosphor-rest or to D-Bus APIs? No, despite their importance, these interfaces are technically internal to the BMC. We might want to document user-impacts to these as well.
- Security fix? Yes, probably, given today's climate
Feel free to come up with your own criteria, based on your estimation of an end-user.

Writing an "end-user impact" statement bridges two worlds. Folks who work on the code may prefer to leave documentation for others or may not fully understand the impact of their change on BMC operations. And nobody else may understand the change. So who should write the end-user impact statement? Anyone who understands the change and how it impacts end-users. That could be developers, testers, or project managers. Multiple people can contribute over time. And it should be okay to document an end-user impact in someone else's change.

It is important that all of the end-user impact statements be "findable". Adding them as comments on selected GitHub issues is an obvious solution, and I suggest using the issue associated with the code change that caused the user impact. (We could use GitHub labels [8][9] to mark issues which have user impact.)

2. Create a change log from end-user impact statements.

Groups who care about end-user impacts include testers, security folks, release managers, and others. They can scan the issues, read the impact statements, and use them to curate a change log [2] which lists all changes that might be interesting to users. Each change log entry should have a brief description of the impact (from the user's point of view) with links back to the issue. The change log could be an openbmc wiki page. (And yes, some people's job includes curating change logs.)

We should have a project-wide change log which captures all impacts. (It is easy to ignore entries you don't care about.) Anyone who cares about a user impact can add items to the change log.

How is the change log useful? Testers will use the change log to help ensure test coverage and add facts about where the test cases are stored and whether the tests passed. Release managers will use it to gauge progress toward goals. Security folks will cover security changes. Technical writers, etc. Each will contribute significant facts about their part of the process, and the log is a place to organize links to demos, etc.

3. Use the change log to make release notes.

The Release Planning work group [3] can refer to the change log as an input to the Release Notes [1][4]. For example, multiple changes can be merged, minor changes can be omitted, important changes can be highlighted, etc. Having good release notes is a hallmark of a mature development process.

Let's be clear about this: Most users will ignore the release notes or use it only to answer specific questions about bugs or features. Only a few will use it to learn more about new functions, changed functions they have to adapt to, how to review test efforts, what security fixes are included, etc. They will make decisions based on its content. For example, some groups may use it as part of a software security assurance process [5].

** Conclusion **

That's the proposal ... create end-user impact statements ... summarize them in a change log ... and massage that into release notes. The real trick is to actually write end-user impact statements as the work is being done. We have the test team [6] and the security work group [7] to help with that, along with pressure from groups who use OpenBMC in their projects.

- Joseph Reynolds

[1]: https://en.m.wikipedia.org/wiki/Release_notes
[2]: https://en.m.wikipedia.org/wiki/Changelog
[3]: https://github.com/openbmc/openbmc/wiki/Release-Planning
[4]: https://github.com/openbmc/docs/blob/master/release/release-notes.md
[5]: https://en.wikipedia.org/wiki/Software_security_assurance
[6]: https://github.com/openbmc/openbmc/wiki/Test-work-group
[7]: https://github.com/openbmc/openbmc/wiki/Security-working-group
[8]: https://help.github.com/en/articles/about-labels
[9]: https://github.com/openbmc/openbmc/labels

#1 lore raw
Quoted 124 lines · Joseph Reynolds

This is a proposal to improve OpenBMC communication and get better release notes.

** Preamble **

Teams that use OpenBMC in other projects will not know OpenBMC as well as the development community, and they will not know what is in each release. The main way for them to find out is by reading the release notes. They will make decisions based on its contents. That makes the release notes an important communication between the development community and its end-users.

We need to get better at communicating new and changed OpenBMC functions. Doing so gives many benefits, including better release notes [1].

** The proposal **

1. Adopt a new practice to provide end-user impact statements when and where it make sense to do so.
2. Begin a project level change log to summarize user impacts.

** Details **

How do these changes help connect the folks who contribute to OpenBMC with the folks who use it? The main points are:
1. Contributors write an "end-user impact" statement to notify users that something they might care about has changed.
2. Community members use end-user impact statements to create a change log which summarizes changes they consider significant.
3. Release planning summarizes the change log as an input to the release notes.

Items in detail:

1. Create end-user impact statements.

An "end-user impact" statement summarizes each change from the point of view of someone familiar with using the external interfaces. It tells them what they need to know about what changed from their point of view, so they know what to test, how to adapt, etc.

When there is an impact, it should have details like:
- Is there an end-user impact (y/n)?
- Is this a new feature or function (where are the docs)?
- Is this a bug fix?
- What does the end-user need to know (one liner)?
These statements don't have to be perfect. Any inkling you can provide that there is a user impact is helpful to the community. The impact might change during the release as the design evolves. Different contributors will see different impacts. That's all normal and okay.

Are the following end-user impacts?
- Change to phosphor-webui that affects the browser display? Answer "no" if it doesn't affect usage, or "Yes" if the user will likely notice or care.
- Bug fix in fan control? Answer depends. Answer "yes" if the bug might start a fire.
- Refactor code? No
- New Redfish API? Yes; new features are positive user impacts.
- Adding new documentation? No; although helpful, docs don't affect the end-user's use of the external interfaces
- BMCWeb retires old and accepts new TLS encryption algorithms? Yes, it affects how you can connect to the BMC
- Changes to phosphor-rest or to D-Bus APIs? No, despite their importance, these interfaces are technically internal to the BMC. We might want to document user-impacts to these as well.
- Security fix? Yes, probably, given today's climate
Feel free to come up with your own criteria, based on your estimation of an end-user.

Writing an "end-user impact" statement bridges two worlds. Folks who work on the code may prefer to leave documentation for others or may not fully understand the impact of their change on BMC operations. And nobody else may understand the change. So who should write the end-user impact statement? Anyone who understands the change and how it impacts end-users. That could be developers, testers, or project managers. Multiple people can contribute over time. And it should be okay to document an end-user impact in someone else's change.

It is important that all of the end-user impact statements be "findable". Adding them as comments on selected GitHub issues is an obvious solution, and I suggest using the issue associated with the code change that caused the user impact. (We could use GitHub labels [8][9] to mark issues which have user impact.)

2. Create a change log from end-user impact statements.

Groups who care about end-user impacts include testers, security folks, release managers, and others. They can scan the issues, read the impact statements, and use them to curate a change log [2] which lists all changes that might be interesting to users. Each change log entry should have a brief description of the impact (from the user's point of view) with links back to the issue. The change log could be an openbmc wiki page. (And yes, some people's job includes curating change logs.)

We should have a project-wide change log which captures all impacts. (It is easy to ignore entries you don't care about.) Anyone who cares about a user impact can add items to the change log.

How is the change log useful? Testers will use the change log to help ensure test coverage and add facts about where the test cases are stored and whether the tests passed. Release managers will use it to gauge progress toward goals. Security folks will cover security changes. Technical writers, etc. Each will contribute significant facts about their part of the process, and the log is a place to organize links to demos, etc.

3. Use the change log to make release notes.

The Release Planning work group [3] can refer to the change log as an input to the Release Notes [1][4]. For example, multiple changes can be merged, minor changes can be omitted, important changes can be highlighted, etc. Having good release notes is a hallmark of a mature development process.

Let's be clear about this: Most users will ignore the release notes or use it only to answer specific questions about bugs or features. Only a few will use it to learn more about new functions, changed functions they have to adapt to, how to review test efforts, what security fixes are included, etc. They will make decisions based on its content. For example, some groups may use it as part of a software security assurance process [5].

Quoted 8 lines

** Conclusion **

That's the proposal ... create end-user impact statements ... summarize them in a change log ... and massage that into release notes. The real trick is to actually write end-user impact statements as the work is being done. We have the test team [6] and the security work group [7] to help with that, along with pressure from groups who use OpenBMC in their projects.

So, I think it's a reasonable proposal, however, I think there needs to be more concrete description of how I as a contributor go about this. There's a sort-of throw-away suggestion above that the user impact descriptions should go in "the github issue associated with the change" - I guess I want to see some agreement that this is how we're going to go about it. You probably also want to start hassling developers to actually write the impact statement; while I don't expect anyone will be opposed to the idea, getting them to take it up is a separate problem (there may also be a problem of having a github issue associated with the change to start with).

Andrew

Quoted 15 lines

- Joseph Reynolds

[1]: https://en.m.wikipedia.org/wiki/Release_notes
[2]: https://en.m.wikipedia.org/wiki/Changelog
[3]: https://github.com/openbmc/openbmc/wiki/Release-Planning
[4]:
https://github.com/openbmc/docs/blob/master/release/release-notes.md
[5]: https://en.wikipedia.org/wiki/Software_security_assurance
[6]: https://github.com/openbmc/openbmc/wiki/Test-work-group
[7]: https://github.com/openbmc/openbmc/wiki/Security-working-group
[8]: https://help.github.com/en/articles/about-labels
[9]: https://github.com/openbmc/openbmc/labels