On the state of documentation

Hi all,

thought I'd try and contribute my thoughts here, as a consumer of OpenBMC rather than a developer on it.

The message has long been that the REST API is what should be used to interact with the BMC, and as such we now have a lot of tooling that does just that (and we're not the only ones).

Over the past $months, it's been frustrating (to say the least) that there hasn't been any good documentation of this API.

Indeed, If I look at the API on the "Rusty's API Scale" of usability:
http://ozlabs.org/~rusty/index.cgi/tech/2008-03-30.html
http://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html
It's all pretty high on the negatives, although it does vary on what specific API you're looking at.

A lot of the negative scoring here is due to a near complete absence of documentation. Most things that we use have ended up being documented either in the implementation (which may or may not allow you to use it correctly), or by some random thread on an IBM internal slack channel.

Additionally, this issue and PR has come up:
"Remove org.openbmc.* from REST server"
https://github.com/openbmc/openbmc/issues/2378
https://gerrit.openbmc-project.xyz/#/c/7422/

This morning, I took a stab at going through the documentation and working out what this means for current documentation:
https://gerrit.openbmc-project.xyz/#/c/7916/

In short: it's not good. There is now no longer documentation even describing how to turn the host on and off.

If the above "remove org.openbmc.* from REST server" is merged, there will be *ZERO* documentation on how to turn the host on and off - which is about the most simple thing you want to do with a BMC.

Is there a plan to attack this?

IMNSHO I'd like to see a standard interface be "the way" to talk to OpenBMC based systems, something like RedFish, which I noticed there was the start of some work on here: https://gerrit.openbmc-project.xyz/#/c/7786/

But even if RedFish/IPMI became *the* way externally, the internal APIs (as in, d-bus APIs) could probably also do with some love.

My initial stab at helping with that is:
https://gerrit.openbmc-project.xyz/#/c/7923/
but even then, that very same API raises a lot of other questions that I haven't addressed - and even in that pull request, I've made a bunch of sweeping statements binding the OpenBMC developers to certain API stability constraints.

Signature · 2 lines

Stewart Smith
OPAL Architect, IBM.

-----Original Message-----
From: openbmc [mailto:openbmc- [email protected]] On Behalf Of Patrick Williams
Sent: Wednesday, November 15, 2017 6:41 AM
To: Stewart Smith <[email protected]>
Cc: OpenBMC Maillist <[email protected]>
Subject: Re: On the state of documentation

The sdbusplus generator that is used on the *-dbus-interfaces repos already supports generating markdown documentation for the APIs. This was suppose to be roughly equivalent to the “old” documentation.

The plan was:
1. Enhance the make files to run the markdown generator.
2. Add a Jenkins job that would run it and publish to the openbmc.github.io
repo on merge to an *-dbus-interfaces repo.

I don’t think either of these steps has been done but they are also not a significant effort.

Have you looked at the current markdown generation output to see if this would be helpful?

Patrick
Sent from my iPhone

Quoted 1 line

On Nov 15, 2017, at 12:11 AM, Stewart Smith

Quoted 19 lines · <[email protected]> wrote:

Hi all,

thought I'd try and contribute my thoughts here, as a consumer of OpenBMC rather than a developer on it.

The message has long been that the REST API is what should be used to interact with the BMC, and as such we now have a lot of tooling that does just that (and we're not the only ones).

Over the past $months, it's been frustrating (to say the least) that there hasn't been any good documentation of this API.

Indeed, If I look at the API on the "Rusty's API Scale" of usability:
http://ozlabs.org/~rusty/index.cgi/tech/2008-03-30.html
http://ozlabs.org/~rusty/index.cgi/tech/2008-04-01.html
It's all pretty high on the negatives, although it does vary on what specific API you're looking at.

Quoted 4 lines

A lot of the negative scoring here is due to a near complete absence of documentation. Most things that we use have ended up being documented either in the implementation (which may or may not allow you to use it correctly), or by some random thread on an IBM internal slack

channel.

Quoted 37 lines

Additionally, this issue and PR has come up:
"Remove org.openbmc.* from REST server"
https://github.com/openbmc/openbmc/issues/2378
https://gerrit.openbmc-project.xyz/#/c/7422/

This morning, I took a stab at going through the documentation and working out what this means for current documentation:
https://gerrit.openbmc-project.xyz/#/c/7916/

In short: it's not good. There is now no longer documentation even describing how to turn the host on and off.

If the above "remove org.openbmc.* from REST server" is merged, there will be *ZERO* documentation on how to turn the host on and off - which is about the most simple thing you want to do with a BMC.

Is there a plan to attack this?

IMNSHO I'd like to see a standard interface be "the way" to talk to OpenBMC based systems, something like RedFish, which I noticed there was the start of some work on here:
https://gerrit.openbmc-project.xyz/#/c/7786/

But even if RedFish/IPMI became *the* way externally, the internal APIs (as in, d-bus APIs) could probably also do with some love.

My initial stab at helping with that is:
https://gerrit.openbmc-project.xyz/#/c/7923/
but even then, that very same API raises a lot of other questions that I haven't addressed - and even in that pull request, I've made a bunch of sweeping statements binding the OpenBMC developers to certain API stability constraints.
--
Stewart Smith
OPAL Architect, IBM.

I believe that the lack of good documentation is a symptom of a larger
issue: OpenBMC is not targeting a public specification with its REST interface. This has led to certain engineering flexibilities with the interface that, while useful for short term to "get something that works", make it very difficult to target openbmc an evolving platform. In the current model, DBus specifications are pushed directly to the end user, which means (nearly) every DBus spec change results in a change to end user interfaces. I don't believe this is sustainable in the long run, given the number of platforms that this is targeting and the need for reproducible APIs between systems.

I would argue that middle to long term we should transition away from these interfaces for end-user facing functionality, and move toward something that has a standard behind it. I think the obvious choice in this problem space is redfish, but I'm open to other options.

I'd like to pose the question:
Should we move toward deprecating the /xyz and /org style rest interfaces, and agree that we should start transitioning to something (redfish or not) with a fixed specification?

Some important key points that I think should be driven by the new model:
1. Rest interfaces should not be driven by the underlying DBus interfaces, nor the backend implementation language.
2. Interfaces should not contain any "backdoor" type APIs that allow white box style access to internal APIs. Anything breaking the black box model should be done in a machine specific layer or configuration.
3. APIs should give the minimum required amount of information to fulfill the end user use case.
4. Interfaces should be backed by the system level security policy (PAM users in the current model).
5. Where possible, the implementations should follow an existing specification. In the cases where the requirements are not supported, the api should follow the style of the supporting interface (Think redfish OEM command support)
6. Documentation changes should be pushed ahead of or along with
implementation support, with the goal of keeping the documentation
constantly up to date.

Shameless plug: As the person that pushed the aforementioned gerrit review for a new thinking on the webserver and redfish, I would love comments on architecture decisions and implementation details.

Quoted 10 lines · Ed

I believe that the lack of good documentation is a symptom of a larger
issue: OpenBMC is not targeting a public specification with its REST interface. This has led to certain engineering flexibilities with the interface that, while useful for short term to "get something that works", make it very difficult to target openbmc an evolving platform. In the current model, DBus specifications are pushed directly to the end user, which means (nearly) every DBus spec change results in a change to end user interfaces. I don't believe this is sustainable in the long run, given the number of platforms that this is targeting and the need for reproducible APIs between systems.

I very much agree.

As someone consuming it, for the purposes of automating testing host firmware (for the purpose of being a host firmware maintainer), it's been *extroadinarily* frustrating. Upgrading a BMC usually meant 0.5-1 day of working out what API change broke and how to code something that was safe for more than one OpenBMC build.

Additionally, much of the docs of the DBUS API are currently written targetting somebody working on OpenBMC internals.

Quoted 1 line

I would argue

Quoted 2 lines

that middle to long term we should transition away from these interfaces for end-user facing functionality, and move toward

Quoted 2 lines

something that has a standard behind it. I think the obvious choice in this problem space is redfish, but I'm open to other options.

I am 100% with you on that.

RedFish may not be ideal (neither is IPMI), but it's *standard*, so at least it's not just a OpenBMC problem.

I see at least 2 other BMCs I have to deal with going the RedFish route, so I'd honestly like to see OpenBMC go there.

I view all the code I've written against the OpenBMC REST API to be throw-away code... but that's what I've been told by the IBM team they support, so there hasn't really been a choice apart from rewriting half of it every month for no discernable good reason.

Quoted 4 lines

I'd like to pose the question:
Should we move toward deprecating the /xyz and /org style rest interfaces, and agree that we should start transitioning to something (redfish or not) with a fixed specification?

My vote is YES, *PLEASE* YES.

Quoted 3 lines

Some important key points that I think should be driven by the new model:
1. Rest interfaces should not be driven by the underlying DBus interfaces, nor the backend implementation language.

Quoted 4 lines

2. Interfaces should not contain any "backdoor" type APIs that allow white box style access to internal APIs. Anything breaking the black box model should be done in a machine specific layer or configuration.
3. APIs should give the minimum required amount of information to fulfill the end user use case.
4. Interfaces should be backed by the system level security policy (PAM users in the current model).
5. Where possible, the implementations should follow an existing specification. In the cases where the requirements are not supported, the api should follow the style of the supporting interface (Think redfish OEM command support)

Quoted 3 lines

6. Documentation changes should be pushed ahead of or along with implementation support, with the goal of keeping the documentation constantly up to date.

Completely agree.

Quoted 3 lines

Shameless plug: As the person that pushed the aforementioned gerrit review for a new thinking on the webserver and redfish, I would love comments on architecture decisions and implementation details.

I nearly commented MERGE NOW without even looking, but that's possibly a bit keen :)

I reckon the Minimum Viable Product for something is enough to do what TripleO needs (which is probably the same set of things that I need for OPAL testing). Once it's there, I say go for it - internals can be cleaned up, but at least those of us consuming OpenBMC have a good way forward along with an upstream spec to point to and look at.

Signature · 2 lines

Stewart Smith
OPAL Architect, IBM.

Quoted 13 lines · Stewart Smith

Quoted 10 lines · Ed

I believe that the lack of good documentation is a symptom of a larger
issue: OpenBMC is not targeting a public specification with its REST interface. This has led to certain engineering flexibilities with the interface that, while useful for short term to "get something that works", make it very difficult to target openbmc an evolving platform. In the current model, DBus specifications are pushed directly to the end user, which means (nearly) every DBus spec change results in a change to end user interfaces. I don't believe this is sustainable in the long run, given the number of platforms that this is targeting and the need for reproducible APIs between systems.

No disagreement here. The project could really could use a Redfish and full IPMI implementation.

Quoted 2 lines

Quoted 2 lines

that middle to long term we should transition away from these interfaces for end-user facing functionality, and move toward

Words like 'transition away' and 'move toward' are kind of freaking me out a little. Are you suggesting that OpenBMC can _only_ have a Redfish implementation? Any other REST API implementation would be…banned from the project? Or am I reading into it too much?

Like I said, I too agree we need a Redfish and IPMI implementation in OpenBMC.

That said, what reason is there to get rid of the existing DBus<->REST translation server? It’s not as if its a maintenance burden. It isn’t as if anyone is making people use it in their products. Its there. You can use it or not use it. When we have an IPMI or Redfish implementation, I’ll be the first in line (well maybe second after Stewart) to tell my employer to turn it off in their products. That isn’t the same thing as removing it from the project altogether.

Quoted 2 lines

Quoted 2 lines

Some important key points that I think should be driven by the new model:
1. Rest interfaces should not be driven by the underlying DBus interfaces, nor the backend implementation language.

This reads like policy for someone using OpenBMC to build a BMC stack.

I would rephrase it to:

XYZ corp cannot use any REST implementations provided by OpenBMC that are driven by the underlying DBus interfaces in XYZ corp products making use of OpenBMC.

Do you see the difference?

Quoted 4 lines

Quoted 4 lines

2. Interfaces should not contain any "backdoor" type APIs that allow white box style access to internal APIs. Anything breaking the black box model should be done in a machine specific layer or configuration.
3. APIs should give the minimum required amount of information to fulfill the end user use case.
4. Interfaces should be backed by the system level security policy (PAM users in the current model).
5. Where possible, the implementations should follow an existing specification. In the cases where the requirements are not supported, the api should follow the style of the supporting interface (Think redfish OEM command support)

I have the same reaction to 2-6 as I did for 1. These are policies on _use_ of OpenBMC. Right? Not what the contributors to the project are allowed to do.

My worldview is that each one of these bullet points would cost the project a potential contributor. Somewhere, someone is going to want to do one of those things. Why not let them, when the only thing you have to do is not use their code?

-----Original Message-----
From: Stewart Smith [mailto:[email protected]]
Sent: Thursday, November 16, 2017 9:11 PM
To: Brad Bishop <[email protected]>; Tanous, Ed <[email protected]>
Cc: OpenBMC Maillist <[email protected]>
Subject: Re: On the state of documentation

Quoted 1 line · Brad Bishop

Quoted 1 line

On Nov 16, 2017, at 10:03 PM, Stewart Smith

Quoted 1 line · <[email protected]> wrote:

Quoted 1 line

Quoted 7 lines

Quoted 7 lines

Quoted 6 lines · Ed

I believe that the lack of good documentation is a symptom of a larger
issue: OpenBMC is not targeting a public specification with its REST interface. This has led to certain engineering flexibilities with the interface that, while useful for short term to "get something that works", make it very difficult to target openbmc an

evolving platform.

Quoted 16 lines

Quoted 5 lines

Quoted 5 lines

In the current model, DBus specifications are pushed directly to the end user, which means (nearly) every DBus spec change results in a change to end user interfaces. I don't believe this is sustainable in the long run, given the number of platforms that this is targeting and the need for reproducible APIs between systems.

No disagreement here. The project could really could use a Redfish and full IPMI implementation.

Quoted 2 lines

Quoted 2 lines

that middle to long term we should transition away from these interfaces for end-user facing functionality, and move toward

Words like 'transition away' and 'move toward' are kind of freaking me out a little. Are you suggesting that OpenBMC can _only_ have a Redfish implementation? Any other REST API implementation would be…banned from the project? Or am I reading into it too much?

I'd support multiple interfaces (after all, IPMI and RedFish are multiple ones anyway, and IPMI is probably going to take a few more years to disappear)

Quoted 10 lines

Like I said, I too agree we need a Redfish and IPMI implementation in OpenBMC.

That said, what reason is there to get rid of the existing DBus<->REST translation server? It’s not as if its a maintenance burden. It isn’t as if anyone is making people use it in their products. Its there. You can use it or not use it. When we have an IPMI or Redfish implementation, I’ll be the first in line (well maybe second after Stewart) to tell my employer to turn it off in their products. That isn’t the same thing as removing it from the project altogether.

It might be useful for debug to keep around "forever"?

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

Some important key points that I think should be driven by the new

model:

Quoted 1 line

Quoted 1 line

Quoted 1 line

1. Rest interfaces should not be driven by the underlying DBus

interfaces, nor the backend implementation language.

Quoted 10 lines

This reads like policy for someone using OpenBMC to build a BMC stack.

I would rephrase it to:

XYZ corp cannot use any REST implementations provided by OpenBMC that are driven by the underlying DBus interfaces in XYZ corp products making use of OpenBMC.

Do you see the difference?

I'd say "The OpenBMC REST API is not meant to be a stable API to be consumed. It exists exclusively for debug and development efforts of OpenBMC. It may, in future, completely change or disappear."

From someone consuming it, that's certainly been the experience at least :)

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

2. Interfaces should not contain any "backdoor" type APIs that allow

white box style access to internal APIs. Anything breaking the black box model should be done in a machine specific layer or configuration.

Quoted 1 line

Quoted 1 line

Quoted 1 line

3. APIs should give the minimum required amount of information to

fulfill the end user use case.

Quoted 1 line

Quoted 1 line

Quoted 1 line

4. Interfaces should be backed by the system level security policy (PAM

users in the current model).

Quoted 7 lines

Quoted 4 lines

Quoted 4 lines

5. Where possible, the implementations should follow an existing specification. In the cases where the requirements are not supported, the api should follow the style of the supporting interface (Think redfish OEM command support)

I have the same reaction to 2-6 as I did for 1. These are policies on _use_ of OpenBMC. Right? Not what the contributors to the project are

allowed to do.

Quoted 5 lines

My worldview is that each one of these bullet points would cost the project a potential contributor. Somewhere, someone is going to want to do one of those things. Why not let them, when the only thing you have to do is not use their code?

Maybe guidelines more than rules?

I'm sure there's exceptions, but there's unlikely to be anything really exist beyond RedFish and IPMI... If there is, I'd say it's up to you if you want to include it in the base/reference distribution and if this will create value or just more work :)

--
Stewart Smith
OPAL Architect, IBM.

Quoted 35 lines

-----Original Message-----
From: Stewart Smith [mailto:[email protected]]
Sent: Thursday, November 16, 2017 9:11 PM
To: Brad Bishop <[email protected]>; Tanous, Ed <[email protected]>
Cc: OpenBMC Maillist <[email protected]>
Subject: Re: On the state of documentation

Quoted 1 line · Brad Bishop

Quoted 1 line

On Nov 16, 2017, at 10:03 PM, Stewart Smith

Quoted 8 lines · <[email protected]> wrote:

Quoted 8 lines

Quoted 6 lines · Ed

I believe that the lack of good documentation is a symptom of a larger
issue: OpenBMC is not targeting a public specification with its REST interface. This has led to certain engineering flexibilities with the interface that, while useful for short term to "get something that works", make it very difficult to target openbmc an

evolving platform.

Quoted 12 lines

Quoted 5 lines

Quoted 5 lines

In the current model, DBus specifications are pushed directly to the end user, which means (nearly) every DBus spec change results in a change to end user interfaces. I don't believe this is sustainable in the long run, given the number of platforms that this is targeting and the need for reproducible APIs between systems.

No disagreement here. The project could really could use a Redfish and full IPMI implementation.

Quoted 2 lines

Quoted 2 lines

that middle to long term we should transition away from these interfaces for end-user facing functionality, and move toward

Quoted 4 lines

Quoted 4 lines

Words like 'transition away' and 'move toward' are kind of freaking me out a little. Are you suggesting that OpenBMC can _only_ have a Redfish implementation? Any other REST API implementation would be…banned from the project? Or am I reading into it too much?

No, I'm in no way arguing that we need to only have one external API. Banning certain APIs or components is something that I don't think fit the needs of the project either.

Overall, I would like new and existing code to be able to treat dbus as an openbmc private API, not an end user facing public API. Said another way, DBus api changes should be possible without changing external system interfaces.

Given the fact that the existing REST API is simply a direct connection to dbus, it seems to stand in the way of this goal.

Long term, I would like to see the existing dbus REST API moved to machine specific layers for the machines that want to use them, and remove it as a reference implementation.

Maybe I'm stepping over the line? Maybe it's too early to start talking transition plane, when the replacement components are in their infancy? I'm not asking for a commitment here, just would like to see what other people's thoughts are.

Quoted 16 lines

I'd support multiple interfaces (after all, IPMI and RedFish are multiple ones anyway, and IPMI is probably going to take a few more years to disappear)

Quoted 10 lines

Like I said, I too agree we need a Redfish and IPMI implementation in OpenBMC.

That said, what reason is there to get rid of the existing DBus<->REST translation server? It’s not as if its a maintenance burden. It isn’t as if anyone is making people use it in their products. Its there. You can use it or not use it. When we have an IPMI or Redfish implementation, I’ll be the first in line (well maybe second after Stewart) to tell my employer to turn it off in their products. That isn’t the same thing as removing it from the project altogether.

Quoted 1 line

It might be useful for debug to keep around "forever"?

For debug, yes, I completely agree that server is useful and we can leave it in the repo, I'm less worried about debug, or giving developers and maintainers the option, more worried about the direct linkage from Dbus to end-user facing APIs, especially in a component that's labeled reference, and added to openbmc builds by default.

Quoted 16 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

Some important key points that I think should be driven by the new

model:

Quoted 1 line

Quoted 1 line

Quoted 1 line

1. Rest interfaces should not be driven by the underlying DBus

interfaces, nor the backend implementation language.

Quoted 10 lines

This reads like policy for someone using OpenBMC to build a BMC stack.

I would rephrase it to:

XYZ corp cannot use any REST implementations provided by OpenBMC that are driven by the underlying DBus interfaces in XYZ corp products making use of OpenBMC.

Do you see the difference?

Fair point. Yes, all these goals are things that Intel needs today for production interfaces. I was trying to see if others have similar requirements. My end goal here is not to prescribe what people are allowed to build, your rewording of the next point seems to render this one moot.

Quoted 1 line

Quoted 5 lines

I'd say "The OpenBMC REST API is not meant to be a stable API to be consumed. It exists exclusively for debug and development efforts of OpenBMC. It may, in future, completely change or disappear."

From someone consuming it, that's certainly been the experience at least :)

That's a much more fair wording. On the other hand, there is a lot of precedence for users relying on "debug" APIs for production functionality, then complaining when the function breaks or disappears. I don't have a solution here, just a comment.

Quoted 14 lines

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

2. Interfaces should not contain any "backdoor" type APIs that allow

white box style access to internal APIs. Anything breaking the black box model should be done in a machine specific layer or configuration.

Quoted 1 line

Quoted 1 line

Quoted 1 line

3. APIs should give the minimum required amount of information to

fulfill the end user use case.

Quoted 1 line

Quoted 1 line

Quoted 1 line

4. Interfaces should be backed by the system level security policy (PAM

users in the current model).

Quoted 5 lines

Quoted 4 lines

Quoted 4 lines

5. Where possible, the implementations should follow an existing specification. In the cases where the requirements are not supported, the api should follow the style of the supporting interface (Think redfish OEM command support)

Quoted 3 lines

Quoted 2 lines

I have the same reaction to 2-6 as I did for 1. These are policies on _use_ of OpenBMC. Right? Not what the contributors to the project are

allowed to do.

Brainstorming for a second; What if each of these were prefixed with "Reference components shall: "? Yes, some of these are a little more prescriptive than I'd like for an open project, but we need some way to convey that certain components are production ready, secure to modern standards, and are supportable long term.

I'd much rather see reference components fall into that category than to create a new system for classification.

With that said, if most of these needs aren't common amongst a majority of the platforms, then I'm happy to do the "XYZ corp" style prescription. Just to be clear, these are thoughts, and learnings pulled from previous production BMC systems I've worked on. I'm certain my biases have bled into some of the points. I was not intending to pose these as new hard rules. Guideline is even a strong word. "What Ed wants to see" might be a better description at this point.

Quoted 1 line

Quoted 1 line

Quoted 6 lines

Quoted 4 lines

My worldview is that each one of these bullet points would cost the project a potential contributor. Somewhere, someone is going to want to do one of those things. Why not let them, when the only thing you have to do is not use their code?

Maybe guidelines more than rules?

I should've made that much more clear. Yes, these were intended to be ideas for how I'd like to see certain interfaces evolve. I had no intention of prescribing what people are allowed to do. I question if I even want to prescribe rules for reference components, but I'd at least have a discussion about it.

Quoted 10 lines

I'm sure there's exceptions, but there's unlikely to be anything really exist beyond RedFish and IPMI... If there is, I'd say it's up to you if you want to include it in the base/reference distribution and if this will create value or just more work :)

--
Stewart Smith
OPAL Architect, IBM.

-----Original Message-----
From: Brad Bishop [mailto:[email protected]]
Sent: Friday, November 17, 2017 10:03 AM
To: Tanous, Ed <[email protected]>
Cc: Stewart Smith <[email protected]>; OpenBMC Maillist <[email protected]>
Subject: Re: On the state of documentation

Quoted 36 lines · Ed

Quoted 35 lines

-----Original Message-----
From: Stewart Smith [mailto:[email protected]]
Sent: Thursday, November 16, 2017 9:11 PM
To: Brad Bishop <[email protected]>; Tanous, Ed <[email protected]>
Cc: OpenBMC Maillist <[email protected]>
Subject: Re: On the state of documentation

Quoted 1 line · Brad Bishop

Quoted 1 line

On Nov 16, 2017, at 10:03 PM, Stewart Smith

Quoted 8 lines · <[email protected]> wrote:

Quoted 8 lines

Quoted 6 lines · Ed

I believe that the lack of good documentation is a symptom of a larger
issue: OpenBMC is not targeting a public specification with its REST interface. This has led to certain engineering flexibilities with the interface that, while useful for short term to "get something that works", make it very difficult to target openbmc an

evolving platform.

Quoted 12 lines

Quoted 5 lines

Quoted 5 lines

In the current model, DBus specifications are pushed directly to the end user, which means (nearly) every DBus spec change results in a change to end user interfaces. I don't believe this is sustainable in the long run, given the number of platforms that this is targeting and the need for reproducible APIs between systems.

No disagreement here. The project could really could use a Redfish and full IPMI implementation.

Quoted 2 lines

Quoted 2 lines

that middle to long term we should transition away from these interfaces for end-user facing functionality, and move toward

Quoted 6 lines

Quoted 4 lines

Quoted 4 lines

Words like 'transition away' and 'move toward' are kind of freaking me out a little. Are you suggesting that OpenBMC can _only_ have a Redfish implementation? Any other REST API implementation would be…banned from the project? Or am I reading into it too much?

No, I'm in no way arguing that we need to only have one external API.

Banning certain APIs or components is something that I don't think fit the needs of the project either.

That's good news. Thanks for clearing that up.

Quoted 1 line

Overall, I would like new and existing code to be able to treat dbus as an

openbmc private API, not an end user facing public API. Said another way, DBus api changes should be possible without changing external system interfaces.

Agreed.

Quoted 1 line

Given the fact that the existing REST API is simply a direct connection to

dbus, it seems to stand in the way of this goal.

Maybe. But this comes down to how the user of OpenBMC is using the direct-connect API.

What if I state it like this? Breaking the external API provided by the direct- connect REST API can never be the rationale for rejecting internal DBus API changes.

Quoted 2 lines

Long term, I would like to see the existing dbus REST API moved to machine

specific layers for the machines that want to use them, and remove it as a reference implementation.

But it is the reference implementation of the direct-connect API.

I want to poke at what it means exactly to move something to a machine specific layer.

- Does it mean the code repository is not in the GitHub openbmc organization namespace?

- Does it mean the recipe for the application is not in meta-phosphor?

- Does it mean the recipe is in meta-phosphor, its just not included in an image by default (explicitly turned on in other layers) ?

- Something else?

Quoted 1 line

Maybe I'm stepping over the line? Maybe it's too early to start talking

transition plane, when the replacement components are in their infancy?

Quoted 1 line

I'm not asking for a commitment here, just would like to see what other

people's thoughts are.

I’m sorry if I’m putting you on the defensive. It's not my intent. The intent is to ask probing questions to help me understand where you are coming from.

Quoted 7 lines

Quoted 7 lines

I'd support multiple interfaces (after all, IPMI and RedFish are multiple ones anyway, and IPMI is probably going to take a few more years to disappear)

Quoted 6 lines

Quoted 6 lines

Quoted 6 lines

Like I said, I too agree we need a Redfish and IPMI implementation in OpenBMC.

That said, what reason is there to get rid of the existing DBus<->REST translation server? It’s not as if its a maintenance burden. It isn’t as if anyone is making people use it in their products. Its

there.

Quoted 7 lines

Quoted 6 lines

Quoted 4 lines

You can use it or not use it. When we have an IPMI or Redfish implementation, I’ll be the first in line (well maybe second after Stewart) to tell my employer to turn it off in their products. That isn’t the same thing as removing it from the project altogether.

It might be useful for debug to keep around "forever"?

For debug, yes, I completely agree that server is useful and we can leave it

in the repo, I'm less worried about debug, or giving developers and maintainers the option, more worried about the direct linkage from Dbus to end-user facing APIs, especially in a component that's labeled reference, and added to openbmc builds by default.

If you submitted a patch that turns it off by default (and turns it back on in all the machine layers, to maintain functional parity with what is there today), I wouldn't expect any push-back.

Quoted 10 lines

Quoted 10 lines

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

Some important key points that I think should be driven by the new

model:

Quoted 1 line

Quoted 1 line

Quoted 1 line

1. Rest interfaces should not be driven by the underlying DBus

interfaces, nor the backend implementation language.

Quoted 3 lines

This reads like policy for someone using OpenBMC to build a BMC stack.

Quoted 8 lines

Quoted 7 lines

Quoted 7 lines

I would rephrase it to:

XYZ corp cannot use any REST implementations provided by OpenBMC that are driven by the underlying DBus interfaces in XYZ corp products making use of OpenBMC.

Do you see the difference?

Fair point. Yes, all these goals are things that Intel needs today for

production interfaces. I was trying to see if others have similar requirements.

Quoted 1 line

My end goal here is not to prescribe what people are allowed to build, your

rewording of the next point seems to render this one moot.

Quoted 8 lines

Quoted 8 lines

I'd say "The OpenBMC REST API is not meant to be a stable API to be consumed. It exists exclusively for debug and development efforts of OpenBMC. It may, in future, completely change or disappear."

From someone consuming it, that's certainly been the experience at least :)

Quoted 1 line

That's a much more fair wording. On the other hand, there is a lot of

precedence for users relying on "debug" APIs for production functionality, then complaining when the function breaks or disappears. I don't have a solution here, just a comment.

Agreed. Is this solvable by turning it off by default and big red warning text somewhere?

Quoted 17 lines

Quoted 17 lines

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

2. Interfaces should not contain any "backdoor" type APIs that allow

white box style access to internal APIs. Anything breaking the black box model should be done in a machine specific layer or configuration.

Quoted 1 line

Quoted 1 line

Quoted 1 line

3. APIs should give the minimum required amount of information to

fulfill the end user use case.

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

4. Interfaces should be backed by the system level security policy (PAM

users in the current model).

Quoted 5 lines

Quoted 4 lines

Quoted 4 lines

5. Where possible, the implementations should follow an existing specification. In the cases where the requirements are not supported, the api should follow the style of the supporting interface (Think redfish OEM command support)

Quoted 6 lines

Quoted 4 lines

Quoted 3 lines

I have the same reaction to 2-6 as I did for 1. These are policies on _use_ of OpenBMC. Right? Not what the contributors to the project are

allowed to do.

Brainstorming for a second; What if each of these were prefixed with

"Reference components shall: "? Yes, some of these are a little more prescriptive than I'd like for an open project, but we need some way to convey that certain components are production ready, secure to modern standards, and are supportable long term.

One thing to keep in mind as we brainstorm is that there is a difference between a component being production ready, secure and supportable, and usage of that component in a larger stack.

For instance we could get to an impeccable implementation of the direct- connect API, with no security bugs, production ready, and easily maintainable. There isn’t a problem until a user decides to include that application in their stack.

Quoted 1 line

I'd much rather see reference components fall into that category than to

create a new system for classification.

Another straw man - What if we split the phosphor layer in two or more?
- meta-phosphor-core
- meta-phosphor-extras

The layers could have different themes. The core layer, in a vacuum, could aspire to have a perfect security model. Something like the direct-connect API would go in a different layer, since its use has potential security ramifications and other badness.

Quoted 3 lines

With that said, if most of these needs aren't common amongst a majority

of the platforms, then I'm happy to do the "XYZ corp" style prescription. Just to be clear, these are thoughts, and learnings pulled from previous production BMC systems I've worked on. I'm certain my biases have bled into some of the points.

Quoted 1 line

I was not intending to pose these as new hard rules. Guideline is even a

strong word. "What Ed wants to see" might be a better description at this point.

Quoted 9 lines

Quoted 8 lines

Quoted 6 lines

My worldview is that each one of these bullet points would cost the project a potential contributor. Somewhere, someone is going to want to do one of those things. Why not let them, when the only thing you have to do is not use their code?

Maybe guidelines more than rules?

I should've made that much more clear. Yes, these were intended to be

ideas for how I'd like to see certain interfaces evolve. I had no intention of prescribing what people are allowed to do. I question if I even want to prescribe rules for reference components, but I'd at least have a discussion about it.

Quoted 11 lines

Quoted 11 lines

I'm sure there's exceptions, but there's unlikely to be anything really exist beyond RedFish and IPMI... If there is, I'd say it's up to you if you want to include it in the base/reference distribution and if this will create value or just more work :)

--
Stewart Smith
OPAL Architect, IBM.

thx - brad