Settings backup and restore tool prototype using Ansible -- RFC on a published project
Hello,
I was thinking about an "ideal" way to allow backing up and restoring the BMC configuration in production and came up with this set of goals:
1. The saved config should be easy to inspect and change;
2. It should not require any additional changes on the BMC side, so should be usable not only with OpenBMC but also with other implementations;
3. The tool usage should be easy to automate;
4. The tool should be easy to integrate into larger management systems and it should scale to arbitrary amount of BMCs;
5. It should be as beneficial as possible to the communities using Free Software.
To my surprise I couldn't find any existing solution so following these ideas I prepared a prototype and published at
https://github.com/paulfertser/ansible_redfish_settings .
Please feel free to comment on approach and the implementation, and of course to provide patches. I have to note I'm not personally herding a bunch of servers so my notions can be quite off, do not hesitate to point out when it doesn't match real life usecases.
Point 1 is followed by saving config in YAML format, that's nice to edit both manually and automatically if needed. Point 2 is fulfilled by doing only Redfish interactions, optional Oem settings can be added conditionally. Points 3 and 4 come naturally from using Ansible which seems to be the most popular system for reproducible mass configuration (and much more), so should be familiar to most BMC operators. Point 5 is helped by repository organisation which makes it natural to submit patches upstream to community.general Ansible collection.
While working on this prototype I contributed few changes for missing functionality in community.general collection, the experience was highly positive as the upstream is welcoming and helpful. So extending the project to handle the settings you care about shouldn't be problematic, see my changes for example:
https://github.com/ansible-collections/community.general/commit/e853bdf6f907872fdd39e8c507061269399942e9
https://github.com/ansible-collections/community.general/commit/129f51cf9d284d107cf197a1805481c2e4645efe
https://github.com/ansible-collections/community.general/commit/abe4e5ce95e6cae4a42629640345e0769c3724a1
Here follows the copy of the project README.md with some more details.
# Description #
This project aims at providing an easy to use mechanism for creating a human-readable and editable BMC settings configuration bundle and for reliable idempotent application of them to an arbitrary number of machines. It should be possible to use it directly or to seamlessly integrate into larger Ansible playbooks as desired. The tasks files are provided under MIT licence terms.
Added as a submodule is a fork of community.general Ansible collection. The changes are minimal: extending backwards compatibility and renaming to not clash with system-wide installation. The idea is to provide fast, easy and straightforward mechanism to prepare and test patches for submitting Github pull requests to upstream
repository: https://github.com/ansible-collections/community.general . After the PRs are merged it's trivial to rebase to get the latest-and-greatest code. Please note this collection is published under GPL-3.0.
The testing was performed against OpenBMC hosts but it should be working reasonably with any other Redfish compliant server.
# System requirements #
Ansible version 2.10.0 (released in 2020) or later.
# Installation instructions #
``` # apt install ansible $ git clone --recursive https://github.com/paulfertser/ansible_redfish_settings ```
If you really need a fully "standalone" solution, consider packaging with Nuitka as described here: https://github.com/HexmosTech/Ansika .
# Usage examples #
Example command to retrieve all the settings from a single host:
``` $ ansible-playbook get_settings.yaml --extra-vars "baseuri=172.41.1.38 username=root password=0penBmc" ```
The data will be saved to settings_172.41.1.38.yaml file.
Example command to set settings for a single host:
``` $ ansible-playbook set_settings.yaml --extra-vars "baseuri=172.41.1.38 username=root password=0penBmc" --extra-vars @settings_172.41.1.38.yaml ```
The repository includes sample inventory.ini which allows to retrieve and set settings en masse:
``` $ ansible-playbook get_settings.yaml -i inventory.ini ```
Signature · 2 lines
Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software! mailto:[email protected]
Some previous discussion on Backup and Restore for reference: https://lore.kernel.org/all/YiEbhAQEdJzf706i@heinlein/T/#m50f639da25e073f973e18b5728c15cc1bf61d168
In summary, the DMTF is open to having a schema proposal submitted, they're just looking for someone from the OpenBMC community to help drive it.
Quoted 119 lines · Paul Fertser
Hello,
I was thinking about an "ideal" way to allow backing up and restoring the BMC configuration in production and came up with this set of goals:
1. The saved config should be easy to inspect and change;
2. It should not require any additional changes on the BMC side, so should be usable not only with OpenBMC but also with other implementations;
3. The tool usage should be easy to automate;
4. The tool should be easy to integrate into larger management systems and it should scale to arbitrary amount of BMCs;
5. It should be as beneficial as possible to the communities using Free Software.
To my surprise I couldn't find any existing solution so following these ideas I prepared a prototype and published at
https://github.com/paulfertser/ansible_redfish_settings .Please feel free to comment on approach and the implementation, and of course to provide patches. I have to note I'm not personally herding a bunch of servers so my notions can be quite off, do not hesitate to point out when it doesn't match real life usecases.
Point 1 is followed by saving config in YAML format, that's nice to edit both manually and automatically if needed. Point 2 is fulfilled by doing only Redfish interactions, optional Oem settings can be added conditionally. Points 3 and 4 come naturally from using Ansible which seems to be the most popular system for reproducible mass configuration (and much more), so should be familiar to most BMC operators. Point 5 is helped by repository organisation which makes it natural to submit patches upstream to community.general Ansible collection.
While working on this prototype I contributed few changes for missing functionality in community.general collection, the experience was highly positive as the upstream is welcoming and helpful. So extending the project to handle the settings you care about shouldn't be problematic, see my changes for example:
https://github.com/ansible-collections/community.general/commit/e853bdf6f907872fdd39e8c507061269399942e9
https://github.com/ansible-collections/community.general/commit/129f51cf9d284d107cf197a1805481c2e4645efe
https://github.com/ansible-collections/community.general/commit/abe4e5ce95e6cae4a42629640345e0769c3724a1Here follows the copy of the project README.md with some more details.
# Description #
This project aims at providing an easy to use mechanism for creating a human-readable and editable BMC settings configuration bundle and for reliable idempotent application of them to an arbitrary number of machines. It should be possible to use it directly or to seamlessly integrate into larger Ansible playbooks as desired. The tasks files are provided under MIT licence terms.
Added as a submodule is a fork of community.general Ansible collection. The changes are minimal: extending backwards compatibility and renaming to not clash with system-wide installation. The idea is to provide fast, easy and straightforward mechanism to prepare and test patches for submitting Github pull requests to upstream
repository: https://github.com/ansible-collections/community.general . After the PRs are merged it's trivial to rebase to get the latest-and-greatest code. Please note this collection is published under GPL-3.0.The testing was performed against OpenBMC hosts but it should be working reasonably with any other Redfish compliant server.
# System requirements #
Ansible version 2.10.0 (released in 2020) or later.
# Installation instructions #
``` # apt install ansible $ git clone --recursive https://github.com/paulfertser/ansible_redfish_settings ```If you really need a fully "standalone" solution, consider packaging with Nuitka as described here: https://github.com/HexmosTech/Ansika .
# Usage examples #
Example command to retrieve all the settings from a single host:
``` $ ansible-playbook get_settings.yaml --extra-vars "baseuri=172.41.1.38 username=root password=0penBmc" ```The data will be saved to
settings_172.41.1.38.yamlfile.Example command to set settings for a single host:
``` $ ansible-playbook set_settings.yaml --extra-vars "baseuri=172.41.1.38 username=root password=0penBmc" --extra-vars @settings_172.41.1.38.yaml ```The repository includes sample
inventory.iniwhich allows to retrieve and set settings en masse:``` $ ansible-playbook get_settings.yaml -i inventory.ini ```Signature · 2 lines
Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software! mailto:[email protected]
Hello Adriana,
Quoted 4 lines · Adriana Kobylak
Some previous discussion on Backup and Restore for
reference: [1]https://lore.kernel.org/all/YiEbhAQEdJzf706i@heinlein/T/#m50f639da25e073f973e18b5728c15cc1bf61d168 In summary, the DMTF is open to having a schema proposal submitted, they're just
looking for someone from the OpenBMC community to help drive it.
That's interesting, thank you. Guess I have seen it before but since the link to the issue was (and still is) 404 it wasn't that informative.
What do you think about my approach though? I would think it can work right away with many different existing Redfish backends implementations without going through standartisation process and covers a wide range of actual usecases. In my first mail I outlined the core ideas which are trivially expandable to proper user stories if needed, so it's not just a "feature" but one can clearly describe why, who, when and where could be using that "what".
Do you have other related and relevant usecases in mind that can not be nicely covered by that prototype I've shown?
Signature · 2 lines
Be free, use free (http://www.gnu.org/philosophy/free-sw.html) software! mailto:[email protected]
Quoted 5 lines · Paul Fertser
[...]
What do you think about my approach though? I would think it can work right away with many different existing Redfish backends implementations without going through standartisation process and covers a wide range of actual usecases. [...]
I just wanted to let the list know that ansible_redfish_settings helped me a great deal in getting a small fleet of ten new servers deployed the other week, even though their BMCs are not equipped with firmware based on OpenBMC at all. I highly recommend taking a close look at Paul's approach if you have to wrangle many BMC configs at once.
Signature · 4 lines
with best regards: - Johannes Truschnigg ( [email protected] ) www: https://johannes.truschnigg.info/