OpenBMC 2.8 security audit results
Here are the results from my "security audit" on the planned OpenBMC 2.8 release. The results are not intended as a complete analysis, but only offer highlights into the BMC's security configuration. Contributions are appreciated. The script to perform these tests was presented here: https://lists.ozlabs.org/pipermail/openbmc/2020-April/021186.html and was followed more-or-less.
- Joseph
__________
The image under test was built from 2.8.0-rc1 and necessarily customized for its target machine. This customization may change some behaviors away from project defaults.
The BMC had its out of box setup, with the password as 0penBmc.
The bmcweb HTTPS certificate was the self-signed cert generated by BMCWeb.
Commands were run one-by-one, and results cut&pasted; any errors in that process are mine.
Not all results are shown here. Some of the results contain sensitive data; these are removed and marked as REDACTED.
$ echo "Collecting test host basic data" $ echo "Checking test host openssl version" Checking test host openssl version $ openssl version OpenSSL 1.0.2k-fips 26 Jan 2017
$ echo "Testing: Ping the BMC" observed: ping works
$ echo "Testing: Check for mDNS discovery service" $ avahi-browse --all --ignore-local --resolve --terminate note: the avahi client was not available result: did not test the BMC's discovery service
$ echo "Testing: network (out of band) IPMI"
$ ipmitool_args="-H ${bmc} -I lanplus -C 17 -U ${bmcadminuser} -P ${bmcadminpassword}"
note: parameter is newly required `-C 17` because cipher suite 3 was removed$ echo "Testing: List IPMI users"
$ ipmitool ${ipmitool_args} user list 1
ID Name Callin Link Auth IPMI Msg Channel Priv Limit
1 root false true true ADMINISTRATOR
2 readonly false true true USER
3 true false false NO ACCESS
4 true false false NO ACCESS
5 true false false NO ACCESS
6 true false false NO ACCESS
7 true false false NO ACCESS
8 true false false NO ACCESS
9 true false false NO ACCESS
10 true false false NO ACCESS
11 true false false NO ACCESS
12 true false false NO ACCESS
13 true false false NO ACCESS
14 true false false NO ACCESS
15 true false false NO ACCESSinterpretation: The default root admin user and an additional readonly user are configured.
$ echo "Testing: Print IPMI network settings"
Testing: Print IPMI network settings
$ ipmitool ${ipmitool_args} lan print 1
Set in Progress : Set Complete
Auth Type Support :
Auth Type Enable : Callback :
: User :
: Operator :
: Admin :
: OEM :
IP Address Source : Static Address
IP Address : REDACTED
Subnet Mask : REDACTED
MAC Address : REDACTED
Default Gateway IP : REDACTED
Default Gateway MAC : 00:00:00:00:00:00
802.1q VLAN ID : Disabled
RMCP+ Cipher Suites : 17
Cipher Suite Priv Max : aaaaaaaaaaaaaaa
: X=Cipher Suite Unused
: c=CALLBACK
: u=USER
: o=OPERATOR
: a=ADMIN
: O=OEM
Bad Password Threshold : Not Availableobservation: Only suite 17 is available; that is as intended
$ echo "Testing: Show IPMI supported cipher suites"
Testing: Show IPMI supported cipher suites
$ ipmitool ${ipmitool_args} channel getciphers ipmi 0x1
ID IANA Auth Alg Integrity Alg Confidentiality Alg
17 N/A hmac_sha256 sha256_128 aes_cbc_128
$ ipmitool ${ipmitool_args} channel getciphers sol 0x1
ID IANA Auth Alg Integrity Alg Confidentiality Alg
17 N/A hmac_sha256 sha256_128 aes_cbc_128Access to the BMC via SSH
jrey@gfwa180:~ $ echo "Testing: security aspects of the SSH connection"
Testing: security aspects of the SSH connection
$ ssh -p ${bmcsshport} -vv ${bmcadminuser}@${bmc} echo "Hello OpenBMC"
OpenSSH_7.8p1, OpenSSL 1.0.1u 22 Sep 2016
...excerpted...snip...
debug1: Local version string SSH-2.0-OpenSSH_7.8
debug1: Remote protocol version 2.0, remote software version dropbear_2019.78
...snip...
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group14-sha256,[email protected]
debug2: host key algorithms: ssh-rsa
debug2: ciphers ctos: aes128-ctr,aes256-ctr
debug2: ciphers stoc: aes128-ctr,aes256-ctr
debug2: MACs ctos: hmac-sha1,hmac-sha2-256
debug2: MACs stoc: hmac-sha1,hmac-sha2-256
debug2: compression ctos: [email protected],none
debug2: compression stoc: [email protected],none
...snip...observation: The BMC offers its exact SSH server version. That is not recommended.
observation: The BMC SSH server offers the algorithms shown above. Should we remove HMAC-SHA1?
$ echo "Testing: SSH SoL connection (port 2200)"
jrey@gfwa180:~ $ timeout 20s ssh -p 2200 ${bmcadminuser}@${bmc}
root@REDACTED's password: <-- entered correct password for BMC root account
result: Access was granted when the correct BMC root account password was used.
result: Access was denied when an incorrect BMC root account password was used.$ echo "Testing: HTTP response headers"
Testing: HTTP response headers
$ curl -k -D ${workdir}/http-response-headers -X GET https://${bmc}/redfish/v1
...the HTTP response body is not shown...
jrey@gfwa180:~ $ cat ${workdir}/http-response-headers
HTTP/1.1 200 OK
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: DENY
Pragma: no-cache
Cache-Control: no-Store,no-Cache
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Security-Policy: default-src 'none'; img-src 'self' data:; font-src 'self'; style-src 'self'; script-src 'self'; connect-src 'self' wss:
Content-Type: application/json
Server: iBMC
Date: Fri, 19 Jun 2020 22:00:00 GMT
Content-Length: 1050observation: the http response appear correct per https://github.com/openbmc/bmcweb/blob/master/include/security_headers_middleware.hpp
$ curl -k -v --tlsv1.1 -X GET https://${bmc}/redfish * About to connect() to 9.41.167.159 port 443 (#0) * Trying 9.41.167.159... * Connected to 9.41.167.159 (9.41.167.159) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb * NSS error -12190 (SSL_ERROR_PROTOCOL_VERSION_ALERT) * Peer reports incompatible or unsupported protocol version. * Closing connection 0 curl: (35) Peer reports incompatible or unsupported protocol version.
observation: TLSv1.1 is disabled as intended
$ curl -k --tlsv1.2 -X GET https://${bmc}/redfish { "v1": "/redfish/v1/" }
observation: TLSv1.2 works as intended
echo "Testing: SSL renegotiation"
$ openssl s_client -connect ${bmc}:443
CONNECTED(00000003)
...snip...
---
R
RENEGOTIATING
140737353955216:error:14094153:SSL routines:ssl3_read_bytes:no renegotiation:s3_pkt.c:1481:observation: Requesting renegotiation (the "R" above) failed. See https://github.com/openbmc/openbmc/issues/3624
# This probes unauthenticated REST API access
$ curl -k -X GET https://${bmc}/ | gunzip <!doctype html><html ng-app="app" ng-csp lang="en"><head><meta http-equiv="Content-Security-Policy"><meta charset="UTF-8"><title>OpenBMC</title><meta name="viewport" content="width=device-width,initial-scale=1"><base href="/"><link rel="shortcut icon" href="/favicon.ico"><link href="/app.css" rel="stylesheet"></head><body ng-style="dataService.bodyStyle" ng-attr-id="{{!dataService.showNavigation ? 'login': ''}}"><app-header ng-if="dataService.showNavigation" path="dataService.path"></app-header><app-navigation ng-if="dataService.showNavigation" path="dataService.path" show-navigation="dataService.showNavigation"></app-navigation><toast ng-if="dataService.showNavigation"></toast><main ng-view ng-class="{'content__container': dataService.showNavigation, 'login__wrapper': !dataService.showNavigation}"></main><script src="/app.bundle.js"></script></body></html>
curl -k -X GET https://${bmc}/redfish/
curl -k -X GET https://${bmc}/redfish/v1/
curl -k -X GET https://${bmc}/redfish/v1/JsonSchemas
note: results omitted
observation: Unauthenticated access to the URIs shown above is allowed, as intended
$ curl -k -X GET https://${bmc}/redfish/v1/Registries Unauthorized $ curl -k -X GET https://${bmc}/redfish/v1/SessionService Unauthorized $ curl -k -X GET https://${bmc}/redfish/v1/SessionService/Sessions Unauthorized $ curl -k -X GET https://${bmc}/redfish/v1/AccountService Unauthorized $ curl -k -X GET https://${bmc}/redfish/v1/AccountService/Accounts Unauthorized
observation: Unauthenticated access to the URIs shown above is rejected, as intended
# Probe BMC security settings via Administrator REST API access : First create a login session:
$ curl --insecure -X POST -D headers.txt \
Quoted 2 lines
https://${bmc}:${bmc_https_port}/redfish/v1/SessionService/Sessions \
-d '{"UserName":"'${bmcadminuser}'", "Password":"'${bmcadminpassword}'"}' | tee${workdir}/results.txt
{
"@odata.id": "/redfish/v1/SessionService/Sessions/lwpuEVQ1th", "@odata.type": "#Session.v1_0_2.Session", "Description": "Manager User Session",
"Id": "REDACTED",
"Name": "User Session",
"UserName": "root"
}
$ authtok=$(grep "^X-Auth-Token: " headers.txt | cut -d' ' -f2 | tr -d '\r')
$ test -n "${authtok}" && echo "Got X-Auth-Token okay (in shell variable authtok)" || { echo "Failed to get X-Auth-Token" && false; }
Got X-Auth-Token okay (in shell variable authtok)
$ sessid=$(grep -e '"Id":' ${workdir}/results.txt | cut -d\" -f4)$ curl -k -H "X-Auth-Token: ${authtok}" -X GET https://${bmc}/redfish/v1/AccountService
{
"@odata.id": "/redfish/v1/AccountService",
"@odata.type": "#AccountService.v1_5_0.AccountService",
"AccountLockoutDuration": 300,
"AccountLockoutThreshold": 5,
"Accounts": {
"@odata.id": "/redfish/v1/AccountService/Accounts"
},
"MaxPasswordLength": 20,
"MinPasswordLength": 7,
"Name": "Account Service",
"Oem": {
"OpenBMC": {
"@odata.type": "#OemAccountService.v1_0_0.AccountService",
"AuthMethods": {
"BasicAuth": true,
"Cookie": true,
"SessionToken": true,
"TLS": false,
"XToken": true
}
}
},
"ServiceEnabled": true
...snip...not all fields are shown...
}note: The AccountLockoutDuration and AccountLockoutThreshold values used for this test may be customized differently than the base OpenBMC image. They appear correct.
observation: the value for MinPasswordLength does not seem to match the content of the BMC's Linux-PAM config file /etc/pam.d/common-password. This should be investigated.
Probe BMC settings when logged in as root
# cat /etc/os-release ID=openbmc-witherspoon NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro)" VERSION="2.8.0-rc1" VERSION_ID=2.8.0-rc1-0-g35a774200 PRETTY_NAME="Phosphor OpenBMC (Phosphor OpenBMC Project Reference Distro) 2.8.0-rc1" BUILD_ID="2.8.0-rc1" ...snip...
The intended release appears to appears to be installed on the BMC.
# cat /etc/shadow root:$6$REDACTED:18425:0:99999:7::: ...snip... readonly:$6$REDACTED:18431:0:99999:7:::
A user account called "readonly" is present.
# groups root priv-admin web redfish ipmi root
note: This denotes the root user has Privilege Role "admin" and Group Roles as shown; these appear correct modulo https://github.com/openbmc/openbmc/issues/3643
observed: User account "readonly" cannot authenticate via ssh (because that requires the Administrator role).
observed: User "testuser" with Role=Administrator can authenticate via SSH.
When logged into the BMC via SSH as testuser:
testuser$ ls -la /home/root
drwxr-xr-x 2 root root 384 Jun 18 00:00 .
drwxr-xr-x 5 root root 368 Jun 20 00:23 ..
-rw------- 1 root root 12437 Jun 20 00:24 .bash_history -rw-r----- 1 root root 459 Jun 19 20:19 bmcweb_persistent_data.json
observation: Non-root user can list files in /home/root; that seems undesireable. The files themselves are not readable.
testuser$ ls -l /etc
...snip...
-rw-r--r-- 1 root root 898 Jun 20 00:23 group ...snip...
-rw------- 1 root root 208 Jun 20 00:23 ipmi_pass ...snip...
-rw------- 1 root root 9 May 26 19:30 key_file ...snip...
drwxr-xr-x 1 root root 248 Jun 19 22:42 pam.d -rw-r--r-- 1 root root 1164 Jun 20 00:23 passwd ...snip...
-r-------- 1 root root 1070 Jun 20 00:23 shadow ...snip...
observation: Files in /etc seem to be protected properly. See CVE-2020-14156 here: https://github.com/openbmc/openbmc/issues/3670
testuser$ ls -lR /etc/ssl/etc/ssl:
drwxr-xr-x 4 root root 296 Jun 10 06:22 certs -rw-r--r-- 1 root root 10909 May 26 18:48 openssl.cnf
/etc/ssl/certs:
drwx------ 2 root root 160 Jun 10 06:22 authority drwx------ 2 root root 304 Jun 10 06:23 https
observation: Certificates under /etc/ssl are protected from reading
Quoted 8 lines · Joseph Reynolds
Here are the results from my "security audit" on the planned OpenBMC 2.8 release. The results are not intended as a complete analysis, but only offer highlights into the BMC's security configuration. Contributions are appreciated. The script to perform these tests was presented here: https://lists.ozlabs.org/pipermail/openbmc/2020-April/021186.html and was followed more-or-less.
$ echo "Checking test host openssl version" Checking test host openssl version $ openssl version OpenSSL 1.0.2k-fips 26 Jan 2017
I'm not sure I get this one - is "test host" a BMC or the one where the test script is being run? If the former, this is really old, no longer publicly supported by the OpenSSL team and has multiple CVEs against it [1], so should be upgraded.
[1] https://www.openssl.org/news/openssl-1.0.2-notes.html
Quoted 1 line
debug1: Remote protocol version 2.0, remote software version dropbear_2019.78
There's a very recent 2020.79, which has one CVE fix and some useful changes (e.g. using getrandom(), AES-GCM and so on), would be good to update it for the next release.
Quoted 1 line
observation: The BMC SSH server offers the algorithms shown above. Should we remove HMAC-SHA1?
While it's not [yet] formally broken in the HMAC setting, IMO SHA1 is "broken enough" to remove it everywhere, so yes, I'd vote for that.
Quoted 10 lines
When logged into the BMC via SSH as testuser:
testuser$ ls -la/home/root
drwxr-xr-x 2 root root 384 Jun 18 00:00 . drwxr-xr-x 5 root root 368 Jun 20 00:23 ..(quoted diff) +0
-2-rw------- 1 root root 12437 Jun 20 00:24 .bash_history -rw-r----- 1 root root 459 Jun 19 20:19bmcweb_persistent_data.json
observation: Non-root user can list files in
/home/root; that seems undesireable. The files themselves are not readable.
Agree, it doesn't seem necessary, so should be restricted
Quoted 5 lines
/etc/ssl/certs:
drwx------ 2 root root 160 Jun 10 06:22 authority drwx------ 2 root root 304 Jun 10 06:23 httpsobservation: Certificates under
/etc/sslare protected from reading
This actually seems to be surplus - *certificates* are public by definition, it's the private parts of them (private keys corresponding to public ones in certificates) that need protection like that.
regards,
Alexander
Quoted 4 lines · Alexander Tereschenko
Quoted 3 lines · Joseph Reynolds
Here are the results from my "security audit" on the planned OpenBMC 2.8 release. The results are not intended as a complete analysis, but only offer highlights into the BMC's security configuration. Contributions are appreciated. The script to perform these tests was presented here: https://lists.ozlabs.org/pipermail/openbmc/2020-April/021186.html and was followed more-or-less.
Quoted 8 lines
Quoted 5 lines
$ echo "Checking test host openssl version" Checking test host openssl version $ openssl version OpenSSL 1.0.2k-fips 26 Jan 2017I'm not sure I get this one - is "test host" a BMC or the one where the test script is being run? If the former, this is really old, no longer publicly supported by the OpenSSL team and has multiple CVEs against it [1], so should be upgraded.
Alexander, thanks for your reply. The "test host" is a Linux system used to probe the BMC via network interfaces such as SSH, HTTPS, and REST APIs. To reflect actual customer use, I used a test host that has an older operating system. I've included the test host's software versions to help answer questions about the results below, when the host version factors into the results. [I'll update my preamble with this information.]
Quoted 5 lines
Quoted 3 lines
debug1: Remote protocol version 2.0, remote software version dropbear_2019.78
There's a very recent 2020.79, which has one CVE fix and some useful changes (e.g. using
getrandom(), AES-GCM and so on), would be good to update it for the next release.
Agreed. Yocto will pull in the new dropbear release, and it will downstream to OpenBMC. We may want to update our dropbear/options.patch
Quoted 21 lines
Quoted 3 lines
observation: The BMC SSH server offers the algorithms shown above. Should we remove HMAC-SHA1?
While it's not [yet] formally broken in the HMAC setting, IMO SHA1 is "broken enough" to remove it everywhere, so yes, I'd vote for that.
Quoted 10 lines
When logged into the BMC via SSH as testuser:
testuser$ ls -la/home/root
drwxr-xr-x 2 root root 384 Jun 18 00:00 . drwxr-xr-x 5 root root 368 Jun 20 00:23 ..(quoted diff) +0
-2-rw------- 1 root root 12437 Jun 20 00:24 .bash_history -rw-r----- 1 root root 459 Jun 19 20:19bmcweb_persistent_data.json
observation: Non-root user can list files in
/home/root; that seems undesireable. The files themselves are not readable.Agree, it doesn't seem necessary, so should be restricted
Quoted 7 lines
Quoted 5 lines
/etc/ssl/certs:
drwx------ 2 root root 160 Jun 10 06:22 authority drwx------ 2 root root 304 Jun 10 06:23 httpsobservation: Certificates under
/etc/sslare protected from readingThis actually seems to be surplus - *certificates* are public by definition, it's the private parts of them (private keys corresponding to public ones in certificates) that need protection like that.
Thanks for the clarification. I've heard a private certificate is improperly being stored along with its public cert in there somewhere, but I don't really know.
Quoted 5 lines
regards,
Alexander
Quoted 10 lines · Joseph Reynolds
Quoted 9 lines · Alexander Tereschenko
Quoted 8 lines · Joseph Reynolds
Here are the results from my "security audit" on the planned OpenBMC 2.8 release. The results are not intended as a complete analysis, but only offer highlights into the BMC's security configuration. Contributions are appreciated. The script to perform these tests was presented here: https://lists.ozlabs.org/pipermail/openbmc/2020-April/021186.html and was followed more-or-less.
$ echo "Checking test host openssl version" Checking test host openssl version $ openssl version OpenSSL 1.0.2k-fips 26 Jan 2017
Quoted 5 lines
Quoted 3 lines
I'm not sure I get this one - is "test host" a BMC or the one where the test script is being run? If the former, this is really old, no longer publicly supported by the OpenSSL team and has multiple CVEs against it [1], so should be upgraded.
Alexander, thanks for your reply. The "test host" is a Linux system used to probe the BMC via network interfaces such as SSH, HTTPS, and REST APIs. To reflect actual customer use, I used a test host that has an older operating system. I've included the test host's software versions to help answer questions about the results below, when the host version factors into the results. [I'll update my preamble with this information.]
Got it, in this case that comment is not important, the text host may have whatever versions :)
Quoted 9 lines
Quoted 7 lines
Quoted 5 lines
/etc/ssl/certs:
drwx------ 2 root root 160 Jun 10 06:22 authority drwx------ 2 root root 304 Jun 10 06:23 httpsobservation: Certificates under
/etc/sslare protected from readingThis actually seems to be surplus - *certificates* are public by definition, it's the private parts of them (private keys corresponding to public ones in certificates) that need protection like that.
Thanks for the clarification. I've heard a private certificate is improperly being stored along with its public cert in there somewhere, but I don't really know.
Ah, indeed, that reminds me - OpenBMC stores both private key and the cert (with the public key) in one file, concatenated. That's how bmcweb consumes that (unlike e.g. Apache or nginx, which have separate settings/paths for private and public keys, though nginx also allows for combining) and indeed that in turn necessitates closing down the respective files/dirs - so your check is correct and my comment is not applicable.
regards,
Alexander