http: add http.sslVerifyStatus to check stapled OCSP responses
Subsystems (2)
PATCH 1/1 http: add http.sslVerifyStatus to check stapled OCSP responses
From: Grayson Gordon <[email protected]>
git asks libcurl to verify the peer certificate and the hostname, but it never sets CURLOPT_SSL_VERIFYSTATUS, so the "Certificate Status Request" TLS extension is never requested and any stapled OCSP response the server does send is ignored.
On an OpenSSL-linked build this is silent. OpenSSL hands the stapled response to the application and takes no view on it:SSL_CTX_set_tlsext_status_cb(3) says the callback "should determine whether the returned OCSP response(s) are acceptable or not", and libcurl only installs that callback when CURLOPT_SSL_VERIFYSTATUS is set. So git will fetch from a server whose own staple says its certificate has been revoked.
A GnuTLS-linked build behaves differently, and the difference does not come from curl. GnuTLS consults a stapled response inside gnutls_certificate_verify_peers(), so the failure surfaces through the verifypeer branch of curl's GnuTLS backend (lib/vtls/gtls.c) whether or not CURLOPT_SSL_VERIFYSTATUS was ever set. The same git, against the same server, therefore enforces revocation or not depending only on how its libcurl was built. That difference is documented here rather than papered
over: this option turns the check on where the backend needs asking, and setting it to false does not turn the check off on GnuTLS.
Add an http.sslVerifyStatus boolean that sets CURLOPT_SSL_VERIFYSTATUS. Because http_options() is the collect_fn of a urlmatch config, the per-URL form works with no further changes:
git config http.https://example.com/.sslVerifyStatus true
It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.
Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483
and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.
If the TLS backend cannot check the staple, curl_easy_setopt() returns CURLE_NOT_BUILT_IN. Fail loudly there rather than carrying on, since silently not checking is precisely what this option exists to prevent.
CURLOPT_SSL_VERIFYSTATUS has been available since libcurl 7.41.0, well below the 7.61.0 floor documented in INSTALL, so no version guard is needed.
The new test exercises the fail-closed path, which needs no CA and no OCSP
responder: lib-httpd's server staples nothing, so enabling the option has to turn a working fetch into a failing one. Verified against an unpatched build, where exactly the two assertions that depend on the new option fail and the three controls still pass, and against OpenSSL, GnuTLS and mbedTLS-linked builds of libcurl.
--- Documentation/config/http.adoc | 17 +++++++ http.c | 21 +++++++++ t/t5567-http-verify-status.sh | 72 +++++++++++++++++++++++++++++++ 3 files changed, 110 insertions(+) create mode 100755 t/t5567-http-verify-status.sh
Documentation/config/http.adoc +17 -0
@@ -196,6 +196,23 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verification +fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. ++ +What it changes depends on the TLS backend libcurl was built against. +An OpenSSL-linked build ignores a stapled response unless this is set. +A GnuTLS-linked build consults the staple during ordinary certificate +verification, so it already rejects a revoked certificate under +`http.sslVerify` alone, and setting this to `false` does not disable +that. Where a backend cannot check the staple at all, git fails with an +error rather than continuing unchecked. + http.sslCert:: File containing the SSL certificate when fetching or pushing over HTTPS. Can be overridden by the `GIT_SSL_CERT` environment
http.c +21 -0
@@ -44,6 +44,7 @@ static CURL *curl_default;char curl_errorstr[CURL_ERROR_SIZE ];static int curl_ssl_verify =-1 ; +static int curl_ssl_verify_status;static int curl_ssl_try;static char *curl_http_version;static char *ssl_cert;
@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))
@@ -1131,6 +1136,22 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +/* +* Ask the TLS backend to check the certificate's revocation +* status via the stapled OCSP response. libcurl defaults this +* off, and no backend except GnuTLS consults the staple on its +* own, so without this git will happily accept a certificate +* whose own staple says it has been revoked. +* +* Off by default because it is fail-closed: a server that +* staples nothing fails verification outright, so enabling it +* globally would break every remote that does not staple. +*/ +if (curl_ssl_verify_status && +curl_easy_setopt (result,CURLOPT_SSL_VERIFYSTATUS ,1L ) !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but the TLS backend of " +"this libcurl cannot verify certificate status" )); +if (curl_http_version) {long opt;if (!get_curl_http_version_opt (curl_http_version, &opt)) {
t/t5567-http-verify-status.sh new file +72 -0
72 lines not on this page · raw message
Signature · 1 line
2.55.0
Quoted 3 lines · graysongordon-gl
CURLOPT_SSL_VERIFYSTATUShas been available since libcurl 7.41.0, well below the 7.61.0 floor documented in INSTALL, so no version guard is needed.
Good to see that the author paid extra attention to compatibility.
http.c static CURL *get_curl_handle(void) · line 1149
Quoted 11 lines
http.c +11
-0+/* +* Ask the TLS backend to check the certificate's revocation +* status via the stapled OCSP response. libcurl defaults this +* off, and no backend except GnuTLS consults the staple on its +* own, so without this git will happily accept a certificate +* whose own staple says it has been revoked. +* +* Off by default because it is fail-closed: a server that +* staples nothing fails verification outright, so enabling it +* globally would break every remote that does not staple. +*/
The comment may not be telling any lies per se, but it is dubious that this belongs here as an in-code comment. Developers hunting a bug they suspect this setting might have caused will need access to this information, and they can access it by running 'git blame' to locate the commit that introduced the code. As long as a solid commit log message explains how you arrived at various design decisions (such as 'off by default because'), they can use that as a starting point. For other developers hunting different bugs or trying to add their own enhancements, the comment is a mere distraction.
http.c static CURL *get_curl_handle(void) · line 1154
Quoted 5 lines
http.c +5
-0+if (curl_ssl_verify_status && +curl_easy_setopt (result,CURLOPT_SSL_VERIFYSTATUS ,1L ) !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but the TLS backend of " +"this libcurl cannot verify certificate status" )); +
Thanks.
PATCH v2 1/1 http: add http.sslVerifyStatus to check stapled OCSP responses
In reply to graysongordon-glFrom: Grayson Gordon <[email protected]>
git asks libcurl to verify the peer certificate and the hostname, but it never sets CURLOPT_SSL_VERIFYSTATUS, so the "Certificate Status Request" TLS extension is never requested and any stapled OCSP response the server does send is ignored.
On an OpenSSL-linked build this is silent. OpenSSL hands the stapled response to the application and takes no view on it:SSL_CTX_set_tlsext_status_cb(3) says the callback "should determine whether the returned OCSP response(s) are acceptable or not", and libcurl only installs that callback when CURLOPT_SSL_VERIFYSTATUS is set. So git will fetch from a server whose own staple says its certificate has been revoked.
A GnuTLS-linked build behaves differently, and the difference does not come from curl. GnuTLS consults a stapled response inside gnutls_certificate_verify_peers(), so the failure surfaces through the verifypeer branch of curl's GnuTLS backend (lib/vtls/gtls.c) whether or not CURLOPT_SSL_VERIFYSTATUS was ever set. The same git, against the same server, therefore enforces revocation or not depending only on how its libcurl was built. That difference is documented here rather than papered
over: this option turns the check on where the backend needs asking, and setting it to false does not turn the check off on GnuTLS.
Add an http.sslVerifyStatus boolean that sets CURLOPT_SSL_VERIFYSTATUS. Because http_options() is the collect_fn of a urlmatch config, the per-URL form works with no further changes:
git config http.https://example.com/.sslVerifyStatus true
It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.
Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483
and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.
If the TLS backend cannot check the staple, curl_easy_setopt() returns CURLE_NOT_BUILT_IN. Fail loudly there rather than carrying on, since silently not checking is precisely what this option exists to prevent.
CURLOPT_SSL_VERIFYSTATUS has been available since libcurl 7.41.0, well below the 7.61.0 floor documented in INSTALL, so no version guard is needed.
The new test exercises the fail-closed path, which needs no CA and no OCSP
responder: lib-httpd's server staples nothing, so enabling the option has to turn a working fetch into a failing one. Verified against an unpatched build, where exactly the two assertions that depend on the new option fail and the three controls still pass, and against OpenSSL, GnuTLS and mbedTLS-linked builds of libcurl.
---
v2: drop the block comment above the setopt. What it explained (why the
check is needed, and why the default is false) is already in the commit message, which is where "git blame" leads anyone debugging this. No code change otherwise.
Documentation/config/http.adoc | 17 +++++++ http.c | 10 ++++ t/t5567-http-verify-status.sh | 72 +++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+) create mode 100755 t/t5567-http-verify-status.sh
Documentation/config/http.adoc +17 -0
@@ -196,6 +196,23 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verification +fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. ++ +What it changes depends on the TLS backend libcurl was built against. +An OpenSSL-linked build ignores a stapled response unless this is set. +A GnuTLS-linked build consults the staple during ordinary certificate +verification, so it already rejects a revoked certificate under +`http.sslVerify` alone, and setting this to `false` does not disable +that. Where a backend cannot check the staple at all, git fails with an +error rather than continuing unchecked. + http.sslCert:: File containing the SSL certificate when fetching or pushing over HTTPS. Can be overridden by the `GIT_SSL_CERT` environment
http.c +10 -0
@@ -44,6 +44,7 @@ static CURL *curl_default;char curl_errorstr[CURL_ERROR_SIZE ];static int curl_ssl_verify =-1 ; +static int curl_ssl_verify_status;static int curl_ssl_try;static char *curl_http_version;static char *ssl_cert;
@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))
@@ -1133,6 +1138,11 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +if (curl_ssl_verify_status && +curl_easy_setopt (result,CURLOPT_SSL_VERIFYSTATUS ,1L ) !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but the TLS backend of " +"this libcurl cannot verify certificate status" )); +if (curl_http_version) {long opt;if (!get_curl_http_version_opt (curl_http_version, &opt)) {
t/t5567-http-verify-status.sh new file +72 -0
72 lines not on this page · raw message
Signature · 1 line
2.55.0
Quoted 15 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git asks libcurl to verify the peer certificate and the hostname, but it never sets
CURLOPT_SSL_VERIFYSTATUS, so the "Certificate Status Request" TLS extension is never requested and any stapled OCSP response the server does send is ignored.On an OpenSSL-linked build this is silent. OpenSSL hands the stapled response to the application and takes no view on it:
SSL_CTX_set_tlsext_status_cb(3)says the callback "should determine whether the returned OCSP response(s) are acceptable or not", and libcurl only installs that callback whenCURLOPT_SSL_VERIFYSTATUSis set. So git will fetch from a server whose own staple says its certificate has been revoked.
Quoted 5 lines
A GnuTLS-linked build behaves differently, and the difference does not come from curl. GnuTLS consults a stapled response inside
gnutls_certificate_verify_peers(), so the failure surfaces through the verifypeer branch of curl's GnuTLS backend (lib/vtls/gtls.c) whether or notCURLOPT_SSL_VERIFYSTATUSwas ever set. The same git, against the same
Nit: this is arguably not the same git, as it links against different libraries. It is not exactly unexpected that using different dependencies may cause different behaviour, even though we should of course try to minimize the differences.
Quoted 15 lines
server, therefore enforces revocation or not depending only on how its libcurl was built. That difference is documented here rather than papered
over: this option turns the check on where the backend needs asking, and setting it to false does not turn the check off on GnuTLS.Add an http.sslVerifyStatus boolean that sets
CURLOPT_SSL_VERIFYSTATUS. Becausehttp_options()is thecollect_fnof a urlmatch config, the per-URL form works with no further changes:git config http.https://example.com/.sslVerifyStatus true
It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.
Quoted 6 lines
Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483
and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.
Okay. One could make the argument that we shouldn't add support for OCSP either if it's being phased out now. But I assume there's still going to be enough servers out there that do use it.
The big question to me is why we want to have this change in the first place. It doesn't help to address the behaviour difference between GnuTLS and OpenSSL: if set to "false" OpenSSL would continue to ignore OCSP, whereas GnuTLS would still honor it. If set to "true", OpenSSL would fail closed, whereas GnuTLS would still behave the same as before. So nothing really changes here, unless I misunderstand something.
We don't really gain security, either, because the setting is disabled by default and can only be enabled host-by-host. I doubt anybody out there is really going to do that though, and consequently we haven't really made the world a more secure place :/
So is there any specific use case that you're after? Who exactly is this new feature for?
Thanks!
Patrick
Quoted 5 lines · graysongordon-gl
Documentation/config/http.adoc | 17 +++++++ http.c | 10 ++++ t/t5567-http-verify-status.sh | 72 +++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+) create mode 100755 t/t5567-http-verify-status.sh
Hmph, if we need a brand new script, please make sure the 4-digit number is not taken, not just in the sources to released versions but by other topics that are in flight.
$ git show origin/seen:t | grep t5567
should be empty, but it is not. It seems mm/lib-httpd-cgi-safe topic grabbed it.
Patrick,
Thank you for reviewing my submission!
I understand the "nit" you're describing: software can't exactly be called the same thing if it is built against different libraries, which in turn creates an opportunity for different behaviors. I agree with your follow-up that we should aim to maintain consistency despite those library choices. The benefits of the Principle of Least Astonishment are well established, and I would argue that users reasonably expect Git to behave consistently regardless of the underlying TLS library.
I can provide additional context to motivate this change. Like you, I currently work for GitLab, but as part of the Professional Services organization, which works directly with customers deploying the software in their environments for production use. I am supporting a government customer whose servers use OCSP stapling. There are many such government and government-adjacent customers that utilize certificates issued by US Department of Defense PKI CAs, which have published policies that explicitly outline support for OCSP:
https://dl.dod.cyber.mil/wp-content/uploads/pki-pke/pdf/Unclass-DoD_X.509_Certificate_Policy_v10.7_Jun_3_21.pdf.
Many DoD PKI CAs serve certificates with stapled OCSP responses today and can reasonably be expected to continue to do so until there is a DoD-wide policy change.
A related bug in GnuTLS has affected my customer in their current production environment, preventing them from being able to push-mirror to repositories on remotes whose servers use OCSP stapling. The push-mirror failure is what prompted my initial investigation into this issue.
Original GnuTLS issue:
https://gitlab.com/gnutls/gnutls/-/work_items/1372, resolved in GnuTLS 3.8.8.
GitLab issue on the Cloud Native GitLab build that resolves this behavior in GitLab's default Helm chart base image:
https://gitlab.com/gitlab-org/build/CNG/-/work_items/2374#note_3653072099
GitLab ships its Helm charts with two primary "flavors": one based on Debian and the other on UBI. On Debian, the default SSL backend is GnuTLS, and the aforementioned issues resolve the problem. On UBI, the default backend is OpenSSL, and this issue surfaces. For my government customers who need FIPS, switching to the UBI-based image is the long-term path forward:
https://gitlab.com/graysongordon-gl/gitaly-tls-experiments/-/blob/main/docs/FIPS-AND-THE-TLS-BACKEND.md?ref_type=heads.
To be more explicit, OpenSSL-linked Git binaries are the default case for many government customers, and those customers frequently interface with Git servers that use this type of certificate revocation mechanism.
In summary, there are many instances of Git servers serving a large base of developers working on government-related software that are impacted by this issue and need this functionality. These users have experienced the pain and confusion of this behavior being broken firsthand in downstream applications and have brought the issue to me. These customers care that their Git clients respect certificate revocation when it occurs, whether from their development machines or through service-to-service communications over Git on platforms like GitLab. They interface with these kinds of certificates frequently and will continue to do so, which warrants the inclusion of this flag. The benefit they would receive is correct validation of a remote's certificate.
As it stands today, users leveraging OpenSSL-linked Git binaries can receive a response indicating that the certificate is valid even when the stapled OCSP response indicates that the certificate has been revoked. I think there is a reasonable case that this could qualify as a low-to-medium severity CVE.
The threat model is:
An attacker steals the private key of a Git server whose certificate is accompanied by an OCSP-stapled response.
The breach is detected, and the certificate authority revokes the certificate. Despite the revocation, Git clients continue to accept the certificate and push/pull code from a malicious Git server impersonating the legitimate server.
A malicious actor could leverage this to facilitate the exfiltration of an organization's Git data.
Similar CVEs against libcurl include:
https://curl.se/mail/lib-2026-04/0036.html,
https://curl.se/docs/CVE-2024-0853.html
In addition, the attacker would need a mechanism for intercepting or redirecting the victim's connection to the Git server, hence this not being a higher-severity issue. However, I think that in the context of DoD systems, this is sufficiently dangerous to warrant remediation, and this patch provides that capability.
On the GitLab side, we already have mechanisms for per-remote configuration values to be passed, and integrating this would not be a monumental lift.
Thank you,
Grayson
Quoted 72 lines · Patrick Steinhardt
Quoted 15 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git asks libcurl to verify the peer certificate and the hostname, but it never sets
CURLOPT_SSL_VERIFYSTATUS, so the "Certificate Status Request" TLS extension is never requested and any stapled OCSP response the server does send is ignored.On an OpenSSL-linked build this is silent. OpenSSL hands the stapled response to the application and takes no view on it:
SSL_CTX_set_tlsext_status_cb(3)says the callback "should determine whether the returned OCSP response(s) are acceptable or not", and libcurl only installs that callback whenCURLOPT_SSL_VERIFYSTATUSis set. So git will fetch from a server whose own staple says its certificate has been revoked.Quoted 5 lines
A GnuTLS-linked build behaves differently, and the difference does not come from curl. GnuTLS consults a stapled response inside
gnutls_certificate_verify_peers(), so the failure surfaces through the verifypeer branch of curl's GnuTLS backend (lib/vtls/gtls.c) whether or notCURLOPT_SSL_VERIFYSTATUSwas ever set. The same git, against the sameNit: this is arguably not the same git, as it links against different libraries. It is not exactly unexpected that using different dependencies may cause different behaviour, even though we should of course try to minimize the differences.
Quoted 15 lines
server, therefore enforces revocation or not depending only on how its libcurl was built. That difference is documented here rather than papered
over: this option turns the check on where the backend needs asking, and setting it to false does not turn the check off on GnuTLS.Add an http.sslVerifyStatus boolean that sets
CURLOPT_SSL_VERIFYSTATUS. Becausehttp_options()is thecollect_fnof a urlmatch config, the per-URL form works with no further changes:git config http.https://example.com/.sslVerifyStatus true
It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.
Quoted 6 lines
Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483
and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.Okay. One could make the argument that we shouldn't add support for OCSP either if it's being phased out now. But I assume there's still going to be enough servers out there that do use it.
The big question to me is why we want to have this change in the first place. It doesn't help to address the behaviour difference between GnuTLS and OpenSSL: if set to "false" OpenSSL would continue to ignore OCSP, whereas GnuTLS would still honor it. If set to "true", OpenSSL would fail closed, whereas GnuTLS would still behave the same as before. So nothing really changes here, unless I misunderstand something.
We don't really gain security, either, because the setting is disabled by default and can only be enabled host-by-host. I doubt anybody out there is really going to do that though, and consequently we haven't really made the world a more secure place :/
So is there any specific use case that you're after? Who exactly is this new feature for?
Thanks!
Patrick
PATCH v3 1/1 http: add http.sslVerifyStatus to check stapled OCSP responses
In reply to Junio C HamanoFrom: Grayson Gordon <[email protected]>
git asks libcurl to verify the peer certificate and the hostname, but it never sets CURLOPT_SSL_VERIFYSTATUS, so the "Certificate Status Request" TLS extension is never requested and any stapled OCSP response the server does send is ignored.
On an OpenSSL-linked build this is silent. OpenSSL hands the stapled response to the application and takes no view on it:SSL_CTX_set_tlsext_status_cb(3) says the callback "should determine whether the returned OCSP response(s) are acceptable or not", and libcurl only installs that callback when CURLOPT_SSL_VERIFYSTATUS is set. So git will fetch from a server whose own staple says its certificate has been revoked.
A GnuTLS-linked build behaves differently, and the difference does not come from curl. GnuTLS consults a stapled response inside gnutls_certificate_verify_peers(), so the failure surfaces through the verifypeer branch of curl's GnuTLS backend (lib/vtls/gtls.c) whether or not CURLOPT_SSL_VERIFYSTATUS was ever set. The same git, against the same server, therefore enforces revocation or not depending only on how its libcurl was built. That difference is documented here rather than papered
over: this option turns the check on where the backend needs asking, and setting it to false does not turn the check off on GnuTLS.
Add an http.sslVerifyStatus boolean that sets CURLOPT_SSL_VERIFYSTATUS. Because http_options() is the collect_fn of a urlmatch config, the per-URL form works with no further changes:
git config http.https://example.com/.sslVerifyStatus true
It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.
Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483
and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.
If the TLS backend cannot check the staple, curl_easy_setopt() returns CURLE_NOT_BUILT_IN. Fail loudly there rather than carrying on, since silently not checking is precisely what this option exists to prevent.
CURLOPT_SSL_VERIFYSTATUS has been available since libcurl 7.41.0, well below the 7.61.0 floor documented in INSTALL, so no version guard is needed.
The new test exercises the fail-closed path, which needs no CA and no OCSP
responder: lib-httpd's server staples nothing, so enabling the option has to turn a working fetch into a failing one. Verified against an unpatched build, where exactly the two assertions that depend on the new option fail and the three controls still pass, and against OpenSSL, GnuTLS and mbedTLS-linked builds of libcurl.
---
v3: rename the test from t5567 to t5568. t5567 is taken on 'seen' by
mm/lib-httpd-cgi-safe. t5568 is free on master, next, seen, jch and maint as of b9720e4723, and sits next to the other http tests. No other change.
v2: drop the block comment above the setopt. What it explained (why the
check is needed, and why the default is false) is already in the commit message, which is where "git blame" leads anyone debugging this. No code change otherwise.
Documentation/config/http.adoc | 17 +++++++ http.c | 10 ++++ t/t5568-http-verify-status.sh | 72 +++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+) create mode 100755 t/t5568-http-verify-status.sh
Documentation/config/http.adoc +17 -0
@@ -196,6 +196,23 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verification +fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. ++ +What it changes depends on the TLS backend libcurl was built against. +An OpenSSL-linked build ignores a stapled response unless this is set. +A GnuTLS-linked build consults the staple during ordinary certificate +verification, so it already rejects a revoked certificate under +`http.sslVerify` alone, and setting this to `false` does not disable +that. Where a backend cannot check the staple at all, git fails with an +error rather than continuing unchecked. + http.sslCert:: File containing the SSL certificate when fetching or pushing over HTTPS. Can be overridden by the `GIT_SSL_CERT` environment
http.c +10 -0
@@ -44,6 +44,7 @@ static CURL *curl_default;char curl_errorstr[CURL_ERROR_SIZE ];static int curl_ssl_verify =-1 ; +static int curl_ssl_verify_status;static int curl_ssl_try;static char *curl_http_version;static char *ssl_cert;
@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))
@@ -1133,6 +1138,11 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +if (curl_ssl_verify_status && +curl_easy_setopt (result,CURLOPT_SSL_VERIFYSTATUS ,1L ) !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but the TLS backend of " +"this libcurl cannot verify certificate status" )); +if (curl_http_version) {long opt;if (!get_curl_http_version_opt (curl_http_version, &opt)) {
t/t5568-http-verify-status.sh new file +72 -0
72 lines not on this page · raw message
Signature · 1 line
2.55.0
Quoted 4 lines · graysongordon-gl
v3: rename the test from t5567 to t5568. t5567 is taken on 'seen' by
mm/lib-httpd-cgi-safe. t5568 is free on master, next, seen, jch and maint as ofb9720e4723, and sits next to the other http tests. No other change.
I thought I first asked whether we need a new script before suggesting moving it out of the way because 't5567' was already taken. It is much better not to waste a scarce, shared resource such as a test number, and doing so avoids breaking the build if we are not careful.
If we really need to add a new script, you would need to squash in at least a patch like this to avoid breaking Meson-based builds.
t/meson.build | 1 + 1 file changed, 1 insertion(+)
t/meson.build +1 -0
@@ -707,6 +707,7 @@ integration_tests = [ 't5564-http-proxy.sh', 't5565-push-multiple.sh', 't5566-push-group.sh', + 't5568-http-verify-status.sh', 't5570-git-daemon.sh', 't5571-pre-push-hook.sh', 't5572-pull-submodule.sh',
Quoted 25 lines · Junio C Hamano
Quoted 4 lines · graysongordon-gl
v3: rename the test from t5567 to t5568. t5567 is taken on 'seen' by
mm/lib-httpd-cgi-safe. t5568 is free on master, next, seen, jch and maint as ofb9720e4723, and sits next to the other http tests. No other change.I thought I first asked whether we need a new script before suggesting moving it out of the way because 't5567' was already taken. It is much better not to waste a scarce, shared resource such as a test number, and doing so avoids breaking the build if we are not careful.
If we really need to add a new script, you would need to squash in at least a patch like this to avoid breaking Meson-based builds.
t/meson.build | 1 + 1 file changed, 1 insertion(+)t/meson.build +0
-0
t/meson.build integration_tests = [ · line 713
Quoted 7 lines
t/meson.build +1
-0@@ -707,6 +707,7 @@ integration_tests = [ 't5564-http-proxy.sh', 't5565-push-multiple.sh', 't5566-push-group.sh', + 't5568-http-verify-status.sh', 't5570-git-daemon.sh', 't5571-pre-push-hook.sh', 't5572-pull-submodule.sh',
BTW, exit status of ls-remote is lost without the following:
t/t5568-http-verify-status.sh +1 -1
@@ -38,7 +38,7 @@ test_expect_success 'create http-accessible bare repository' ' # server makes the tests below fail loudly instead of silently vanishing. test_lazy_prereq SSL_VERIFYSTATUS ' git -c http.sslVerifyStatus=true \ - ls-remote "$HTTPD_URL/smart/repo.git" 2>err + ls-remote "$HTTPD_URL/smart/repo.git" 2>err && ! grep "cannot verify certificate status" err '
PATCH v4 1/1 http: add http.sslVerifyStatus to check stapled OCSP responses
From: Grayson Gordon <[email protected]>
git asks libcurl to verify the peer certificate and the hostname, but it never sets CURLOPT_SSL_VERIFYSTATUS, so the "Certificate Status Request" TLS extension is never requested and any stapled OCSP response the server does send is ignored.
On an OpenSSL-linked build this is silent. OpenSSL hands the stapled response to the application and takes no view on it:SSL_CTX_set_tlsext_status_cb(3) says the callback "should determine whether the returned OCSP response(s) are acceptable or not", and libcurl only installs that callback when CURLOPT_SSL_VERIFYSTATUS is set. So git will fetch from a server whose own staple says its certificate has been revoked.
A GnuTLS-linked build behaves differently, and the difference does not come from curl. GnuTLS consults a stapled response inside gnutls_certificate_verify_peers(), so the failure surfaces through the verifypeer branch of curl's GnuTLS backend (lib/vtls/gtls.c) whether or not CURLOPT_SSL_VERIFYSTATUS was ever set. The same git, against the same server, therefore enforces revocation or not depending only on how its libcurl was built. That difference is documented here rather than papered
over: this option turns the check on where the backend needs asking, and setting it to false does not turn the check off on GnuTLS.
Add an http.sslVerifyStatus boolean that sets CURLOPT_SSL_VERIFYSTATUS. Because http_options() is the collect_fn of a urlmatch config, the per-URL form works with no further changes:
git config http.https://example.com/.sslVerifyStatus true
It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.
Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483
and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.
If the TLS backend cannot check the staple, curl_easy_setopt() returns CURLE_NOT_BUILT_IN. Fail loudly there rather than carrying on, since silently not checking is precisely what this option exists to prevent.
CURLOPT_SSL_VERIFYSTATUS has been available since libcurl 7.41.0, well below the 7.61.0 floor documented in INSTALL, so no version guard is needed.
The tests go in t5551 and run only in its https pass, which t5559 provides by sourcing t5551 with LIB_HTTPD_SSL set; that is the only https server the suite has. They exercise the fail-closed path, which needs no CA and no OCSP responder: lib-httpd's server staples nothing, so enabling the option has to turn a working fetch into a failing one. Verified against an unpatched build, where exactly the two assertions that depend on the new option fail and the two controls still pass, and against OpenSSL, GnuTLS and mbedTLS-linked builds of libcurl.
---
v4: drop the new test script. The four tests now live in t5551, keyed on $HTTPD_PROTO so they run in the https pass (t5559) only.
To answer the question I skipped past in v3: no, we do not need a new script. t5559 is t5551 run with LIB_HTTPD_SSL set, and it is the only https server the suite has, so a new script would have had to stand up a second one to reach the same place. Nothing is left over from t5567 or t5568 and no test number is spent.
That also means the t/meson.build hunk is not squashed in. t5551 is already listed there. Adding t5568 to the list now would break configure the other way round, since the list is checked against ls in both directions and errors with "Test files configured, but not found".
On the lost exit status: changed, but to test_might_fail rather than a bare &&. The ls-remote in the prerequisite is expected to fail, that is the premise of the test, so && short-circuits on the expected failure and leaves the prerequisite unsatisfied. Run against the https server both ways:
bare && ok 51 # skip http.sslVerifyStatus=true fails
without a staple (missing SSL_VERIFYSTATUS)
test_might_fail ok 51 - http.sslVerifyStatus=true fails
without a staple
The first still reports "passed all 61 test(s)", which is the failure mode the prerequisite was written to avoid. test_might_fail keeps the chain intact and says the status is ignored on purpose.
Also dropped the "ls-remote succeeds with http.sslVerifyStatus unset" test. It was a control for the standalone script, and in t5551 the surrounding tests already exercise that URL throughout.
The rationale that sat in an in-code comment in v3 is in the log now, per the earlier review.
Verified: t5551 over plain http and t5559 over https both pass all 61 tests, with the four new ones skipping on the former and running on the latter.
Documentation/config/http.adoc | 17 +++++++++++++++++ http.c | 10 ++++++++++ t/t5551-http-fetch-smart.sh | 29 +++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+)
Documentation/config/http.adoc +17 -0
@@ -196,6 +196,23 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verification +fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. ++ +What it changes depends on the TLS backend libcurl was built against. +An OpenSSL-linked build ignores a stapled response unless this is set. +A GnuTLS-linked build consults the staple during ordinary certificate +verification, so it already rejects a revoked certificate under +`http.sslVerify` alone, and setting this to `false` does not disable +that. Where a backend cannot check the staple at all, git fails with an +error rather than continuing unchecked. + http.sslCert:: File containing the SSL certificate when fetching or pushing over HTTPS. Can be overridden by the `GIT_SSL_CERT` environment
http.c +10 -0
@@ -44,6 +44,7 @@ static CURL *curl_default;char curl_errorstr[CURL_ERROR_SIZE ];static int curl_ssl_verify =-1 ; +static int curl_ssl_verify_status;static int curl_ssl_try;static char *curl_http_version;static char *ssl_cert;
@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))
@@ -1133,6 +1138,11 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +if (curl_ssl_verify_status && +curl_easy_setopt (result,CURLOPT_SSL_VERIFYSTATUS ,1L ) !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but the TLS backend of " +"this libcurl cannot verify certificate status" )); +if (curl_http_version) {long opt;if (!get_curl_http_version_opt (curl_http_version, &opt)) {
t/t5551-http-fetch-smart.sh +29 -0
@@ -680,6 +680,35 @@ test_expect_success 'passing hostname resolution information works' ' git -c "http.curloptResolve=$BOGUS_HOST:$LIB_HTTPD_PORT:127.0.0.1" ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null ' +test_lazy_prereq SSL_VERIFYSTATUS ' + test "$HTTPD_PROTO" = "https" && + test_might_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" 2>err && + ! grep "cannot verify certificate status" err +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=true fails without a staple' ' + test_must_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=false is a no-op' ' + git -c http.sslVerifyStatus=false \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus applies to a matching URL' ' + test_must_fail git -c "http.$HTTPD_URL/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus is not applied to other URLs' ' + git -c "http.https://example.com/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + # here user%40host is the URL-encoded version of user@host, # which is our intentionally-odd username to catch parsing errors url_user=$HTTPD_URL_USER/auth/smart/repo.git
Signature · 1 line
2.50.1 (Apple Git-155)
Quoted 7 lines · graysongordon-gl
Verified: t5551 over plain http and t5559 over https both pass all 61 tests, with the four new ones skipping on the former and running on the latter.
Documentation/config/http.adoc | 17 +++++++++++++++++ http.c | 10 ++++++++++ t/t5551-http-fetch-smart.sh | 29 +++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+)
OK, instead of adding a new test script that weighs 72-line we are testing the feature with 29-line addition, which sounds like a good economy ;-).
The code changes and the documentation haven't changed since the previous round, both looking good.
Will replace. Thanks.
Quoted 103 lines
Documentation/config/http.adoc +17
-0@@ -196,6 +196,23 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verification +fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. ++ +What it changes depends on the TLS backend libcurl was built against. +An OpenSSL-linked build ignores a stapled response unless this is set. +A GnuTLS-linked build consults the staple during ordinary certificate +verification, so it already rejects a revoked certificate under +`http.sslVerify` alone, and setting this to `false` does not disable +that. Where a backend cannot check the staple at all, git fails with an +error rather than continuing unchecked. + http.sslCert:: File containing the SSL certificate when fetching or pushing over HTTPS. Can be overridden by the `GIT_SSL_CERT` environmenthttp.c +10
-0@@ -44,6 +44,7 @@ static CURL *curl_default;char curl_errorstr[CURL_ERROR_SIZE ];static int curl_ssl_verify =-1 ; +static int curl_ssl_verify_status;static int curl_ssl_try;static char *curl_http_version;static char *ssl_cert;@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))@@ -1133,6 +1138,11 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +if (curl_ssl_verify_status && +curl_easy_setopt (result,CURLOPT_SSL_VERIFYSTATUS ,1L ) !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but the TLS backend of " +"this libcurl cannot verify certificate status" )); +if (curl_http_version) {long opt;if (!get_curl_http_version_opt (curl_http_version, &opt)) {t/t5551-http-fetch-smart.sh +29
-0@@ -680,6 +680,35 @@ test_expect_success 'passing hostname resolution information works' ' git -c "http.curloptResolve=$BOGUS_HOST:$LIB_HTTPD_PORT:127.0.0.1" ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null ' +test_lazy_prereq SSL_VERIFYSTATUS ' + test "$HTTPD_PROTO" = "https" && + test_might_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" 2>err && + ! grep "cannot verify certificate status" err +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=true fails without a staple' ' + test_must_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=false is a no-op' ' + git -c http.sslVerifyStatus=false \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus applies to a matching URL' ' + test_must_fail git -c "http.$HTTPD_URL/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus is not applied to other URLs' ' + git -c "http.https://example.com/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + # here user%40host is the URL-encoded version of user@host, # which is our intentionally-odd username to catch parsing errors url_user=$HTTPD_URL_USER/auth/smart/repo.git
Quoted 15 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git asks libcurl to verify the peer certificate and the hostname, but it never sets
CURLOPT_SSL_VERIFYSTATUS, so the "Certificate Status Request" TLS extension is never requested and any stapled OCSP response the server does send is ignored.On an OpenSSL-linked build this is silent. OpenSSL hands the stapled response to the application and takes no view on it:
SSL_CTX_set_tlsext_status_cb(3)says the callback "should determine whether the returned OCSP response(s) are acceptable or not", and libcurl only installs that callback whenCURLOPT_SSL_VERIFYSTATUSis set. So git will fetch from a server whose own staple says its certificate has been revoked.
Quoted 9 lines
A GnuTLS-linked build behaves differently, and the difference does not come from curl. GnuTLS consults a stapled response inside
gnutls_certificate_verify_peers(), so the failure surfaces through the verifypeer branch of curl's GnuTLS backend (lib/vtls/gtls.c) whether or notCURLOPT_SSL_VERIFYSTATUSwas ever set. The same git, against the same server, therefore enforces revocation or not depending only on how its libcurl was built. That difference is documented here rather than papered
over: this option turns the check on where the backend needs asking, and setting it to false does not turn the check off on GnuTLS.
This is only part of the story though: GnuTLS 3.8 introduced GNUTLS_NO_STATUS_REQUEST, and curl 8.10 started to set that option in case of !verifystatus. So with new-enough versions of both libraries, Git behaves the same no matter whether we use OpenSSL or GnuTLS as backend. See also aeb1a281ca (gtls: fix OCSP stapling management, 2024-08-20) in curl.
Quoted 10 lines
Add an http.sslVerifyStatus boolean that sets
CURLOPT_SSL_VERIFYSTATUS. Becausehttp_options()is thecollect_fnof a urlmatch config, the per-URL form works with no further changes:git config http.https://example.com/.sslVerifyStatus true
It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.
Quoted 6 lines
Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483
and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.
But... don't we still leave the default to libcurl? If "http.sslVerifyStatus" is not set then we don't touch CURLOPT_SSL_VERIFYSTATUS, either.
I might be misreading this though, as the whole commit message is quite hard to digest. I'd assume that this is because it's generated by AI, and it added a lot of the usual weird phrases to the message. It might be a good idea to adapt the message to have a bit more of a human touch to it.
Quoted 3 lines
If the TLS backend cannot check the staple,
curl_easy_setopt()returnsCURLE_NOT_BUILT_IN. Fail loudly there rather than carrying on, since silently not checking is precisely what this option exists to prevent.
Makes sense.
Quoted 14 lines
Documentation/config/http.adoc +6
-0@@ -196,6 +196,23 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verification
Documentation/config/http.adoc http.sslVerify:: · line 214
Quoted 10 lines
Documentation/config/http.adoc +10
-0+fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. ++ +What it changes depends on the TLS backend libcurl was built against. +An OpenSSL-linked build ignores a stapled response unless this is set. +A GnuTLS-linked build consults the staple during ordinary certificate +verification, so it already rejects a revoked certificate under +`http.sslVerify` alone, and setting this to `false` does not disable +that. Where a backend cannot check the staple at all, git fails with an +error rather than continuing unchecked.
This information is not accurate because recent GnuTLS+libcurl versions handle this the same as OpenSSL, as mentioned above.
Also, it might make sense to convert the backend-specific information into a bulleted list as we may add more items to it going forward. Do we have any info how other backends like mbedTLS behave? Or do we know that those all fail.
Quoted 16 lines
http.c +4
-0@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))
http.c static CURL *get_curl_handle(void) · line 1144
Quoted 7 lines
http.c +4
-0@@ -1133,6 +1138,11 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +if (curl_ssl_verify_status && +curl_easy_setopt (result,CURLOPT_SSL_VERIFYSTATUS ,1L ) !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but the TLS backend of " +"this libcurl cannot verify certificate status" ));
Should we include the output of curl_easy_strerror() in the error message? That'd cause us to include the following error message in case we see CURLE_NOT_BUILT_IN:
case CURLE_NOT_BUILT_IN:
return "A requested feature, protocol or option was not found built-in in"
" this libcurl due to a build-time decision.";
So we could instead do:
if (curl_ssl_verify_status) {
CURLcode error = curl_easy_setopt(result, CURLOPT_SSL_VERIFYSTATUS, 1L);
if (error != CURLE_OK)
die(_("http.sslVerifyStatus is set, but could not enable OCSP status verification: %s"),
curl_easy_strerror(error));
}Quoted 26 lines
t/t5551-http-fetch-smart.sh +18
-0@@ -680,6 +680,35 @@ test_expect_success 'passing hostname resolution information works' ' git -c "http.curloptResolve=$BOGUS_HOST:$LIB_HTTPD_PORT:127.0.0.1" ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null ' +test_lazy_prereq SSL_VERIFYSTATUS ' + test "$HTTPD_PROTO" = "https" && + test_might_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" 2>err && + ! grep "cannot verify certificate status" err +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=true fails without a staple' ' + test_must_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=false is a no-op' ' + git -c http.sslVerifyStatus=false \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' +
t/t5551-http-fetch-smart.sh test_expect_success 'passing hostname resolution information works' ' · line 710
Quoted 10 lines
t/t5551-http-fetch-smart.sh +10
-0+test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus applies to a matching URL' ' + test_must_fail git -c "http.$HTTPD_URL/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus is not applied to other URLs' ' + git -c "http.https://example.com/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +'
Can we reasonably add tests that send OCSP information and verify that enabling "sslVerifyStatus" makes this work as expected?
Patrick
Patrick,
Thanks again for the feedback. I'm going to break this into sections delimited by all caps headers to address each thing you mentioned.
ON GNUTLS VS OPENSSL DIFFERENCES
I appreciate you including the extra context around GnuTLS 3.8's GNUTLS_NO_STATUS_REQUEST flag, and curl 8.10 setting it when "verifystatus" is false.
Fair enough, said another way, if either of these are true:
Condition 1: curl is being built with GnuTLS version < 3.8. (There's no NO_STATUS_REQUEST flag to set.)
OR
Condition 2: curl version < 8.10. (Curl's not using the flag.)
You'll see a discrepancy in cert verification behavior between the versions of git built with GnuTLS vs OpenSSL.
While I appreciate the increased precision here, the purpose of my contribution was to expose the functionality that enables git users to set it if they so choose. As it stands today, this option is not presented. So while the discrepancy is what incited me to look deeper, it isn't the central reason I'm here.
You had a section further down that said: "This information is not accurate because recent GnuTLS+libcurl versions
handle this the same as OpenSSL, as mentioned above." I think that this response section suffices for both.
I didn't look into any other TLS backends, as curl's docs say that the verifystatus option only works with GnuTLS and OpenSSL
https://curl.se/libcurl/c/CURLOPT_SSL_VERIFYSTATUS.html and the other backend options didn't apply to GitLab customers as it relates to our CNG charts. There may still be value in looking and capturing it in the git docs somewhere.
---------------
ON DEFAULT BEHAVIOR AND THE COMMIT MESSAGE
In reality we ARE leaving the default to curl without the flag being set, so my comment "Leaving the default to libcurl is not an option either." wasn't super precise either. A nit.
Again the change that's being introduced here is an OPTION for git users to include the "verifystatus" flag.
Sorry that the commit message feels a bit awkward to you, I can rework it to be a bit more direct and not spend as much time on justifications if that'll be easier to read.
---------------
ON DESCRIPTIVE ERROR MESSAGES
Yes, I like this idea. We could give user's a much clearer error that way. I'll include that. My tests grep for the current string though, so I'll have to update that too.
---------------
ON COMPREHENSIVE TESTING
I'll leave this at you and Junio's discretion. I worked with him earlier in this thread to avoid introducing another test file and keep the testing succinct.
Right now these tests are just limited to parsing the config and applying it to the user-provided remote.
We COULD do the full suite of tests that cover the full range of cases/behaviors:
- The flag is set AND
- no staple sent (should fail)
- good staple (should pass)
- bad staple (should fail)
etc.
We're going to need a lot of infrax for that though:
- test CA.
- test server certificate issued by that CA.
- OCSP responder which knows the certificate's status.
- a way for the TLS server to obtain and staple that response.
- a way to control the response so you can test good vs revoked/invalid.
I set all of that stuff up in my own experiment repo, emulating this with nginx in docker...
It's feasible, just need to know how you all would like it.
- Grayson
Quoted 182 lines · Patrick Steinhardt
Quoted 15 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git asks libcurl to verify the peer certificate and the hostname, but it never sets
CURLOPT_SSL_VERIFYSTATUS, so the "Certificate Status Request" TLS extension is never requested and any stapled OCSP response the server does send is ignored.On an OpenSSL-linked build this is silent. OpenSSL hands the stapled response to the application and takes no view on it:
SSL_CTX_set_tlsext_status_cb(3)says the callback "should determine whether the returned OCSP response(s) are acceptable or not", and libcurl only installs that callback whenCURLOPT_SSL_VERIFYSTATUSis set. So git will fetch from a server whose own staple says its certificate has been revoked.Quoted 9 lines
A GnuTLS-linked build behaves differently, and the difference does not come from curl. GnuTLS consults a stapled response inside
gnutls_certificate_verify_peers(), so the failure surfaces through the verifypeer branch of curl's GnuTLS backend (lib/vtls/gtls.c) whether or notCURLOPT_SSL_VERIFYSTATUSwas ever set. The same git, against the same server, therefore enforces revocation or not depending only on how its libcurl was built. That difference is documented here rather than papered
over: this option turns the check on where the backend needs asking, and setting it to false does not turn the check off on GnuTLS.This is only part of the story though: GnuTLS 3.8 introduced
GNUTLS_NO_STATUS_REQUEST, and curl 8.10 started to set that option in case of!verifystatus. So with new-enough versions of both libraries, Git behaves the same no matter whether we use OpenSSL or GnuTLS as backend. See alsoaeb1a281ca(gtls: fix OCSP stapling management, 2024-08-20) in curl.Quoted 10 lines
Add an http.sslVerifyStatus boolean that sets
CURLOPT_SSL_VERIFYSTATUS. Becausehttp_options()is thecollect_fnof a urlmatch config, the per-URL form works with no further changes:git config http.https://example.com/.sslVerifyStatus true
It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.
Quoted 6 lines
Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483
and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.But... don't we still leave the default to libcurl? If "http.sslVerifyStatus" is not set then we don't touch
CURLOPT_SSL_VERIFYSTATUS, either.I might be misreading this though, as the whole commit message is quite hard to digest. I'd assume that this is because it's generated by AI, and it added a lot of the usual weird phrases to the message. It might be a good idea to adapt the message to have a bit more of a human touch to it.
Quoted 3 lines
If the TLS backend cannot check the staple,
curl_easy_setopt()returnsCURLE_NOT_BUILT_IN. Fail loudly there rather than carrying on, since silently not checking is precisely what this option exists to prevent.Makes sense.
Quoted 14 lines
Documentation/config/http.adoc +6
-0@@ -196,6 +196,23 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verificationDocumentation/config/http.adoc http.sslVerify:: · line 214
Quoted 10 lines
Documentation/config/http.adoc +10
-0+fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. ++ +What it changes depends on the TLS backend libcurl was built against. +An OpenSSL-linked build ignores a stapled response unless this is set. +A GnuTLS-linked build consults the staple during ordinary certificate +verification, so it already rejects a revoked certificate under +`http.sslVerify` alone, and setting this to `false` does not disable +that. Where a backend cannot check the staple at all, git fails with an +error rather than continuing unchecked.This information is not accurate because recent GnuTLS+libcurl versions handle this the same as OpenSSL, as mentioned above.
Also, it might make sense to convert the backend-specific information into a bulleted list as we may add more items to it going forward. Do we have any info how other backends like mbedTLS behave? Or do we know that those all fail.
Quoted 16 lines
http.c +4
-0@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))http.c static CURL *get_curl_handle(void) · line 1144
Quoted 7 lines
http.c +4
-0@@ -1133,6 +1138,11 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +if (curl_ssl_verify_status && +curl_easy_setopt (result,CURLOPT_SSL_VERIFYSTATUS ,1L ) !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but the TLS backend of " +"this libcurl cannot verify certificate status" ));Should we include the output of
curl_easy_strerror()in the error message? That'd cause us to include the following error message in case we seeCURLE_NOT_BUILT_IN:case
CURLE_NOT_BUILT_IN:
return "A requested feature, protocol or option was not found built-in in"
" this libcurl due to a build-time decision.";So we could instead do:
if (curl_ssl_verify_status) { CURLcode error = curl_easy_setopt(result, CURLOPT_SSL_VERIFYSTATUS, 1L); if (error != CURLE_OK) die(_("http.sslVerifyStatus is set, but could not enable OCSP status verification: %s"), curl_easy_strerror(error)); }Quoted 26 lines
t/t5551-http-fetch-smart.sh +18
-0@@ -680,6 +680,35 @@ test_expect_success 'passing hostname resolution information works' ' git -c "http.curloptResolve=$BOGUS_HOST:$LIB_HTTPD_PORT:127.0.0.1" ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null ' +test_lazy_prereq SSL_VERIFYSTATUS ' + test "$HTTPD_PROTO" = "https" && + test_might_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" 2>err && + ! grep "cannot verify certificate status" err +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=true fails without a staple' ' + test_must_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=false is a no-op' ' + git -c http.sslVerifyStatus=false \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' +t/t5551-http-fetch-smart.sh test_expect_success 'passing hostname resolution information works' ' · line 710
Quoted 10 lines
t/t5551-http-fetch-smart.sh +10
-0+test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus applies to a matching URL' ' + test_must_fail git -c "http.$HTTPD_URL/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus is not applied to other URLs' ' + git -c "http.https://example.com/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +'Can we reasonably add tests that send OCSP information and verify that enabling "sslVerifyStatus" makes this work as expected?
Patrick
Quoted 6 lines · Patrick Steinhardt
This is only part of the story though: GnuTLS 3.8 introduced
GNUTLS_NO_STATUS_REQUEST, and curl 8.10 started to set that option in case of!verifystatus. So with new-enough versions of both libraries, Git behaves the same no matter whether we use OpenSSL or GnuTLS as backend. See alsoaeb1a281ca(gtls: fix OCSP stapling management, 2024-08-20) in curl.
Thanks for additional details.
Quoted 21 lines
Quoted 10 lines
Add an http.sslVerifyStatus boolean that sets
CURLOPT_SSL_VERIFYSTATUS. Becausehttp_options()is thecollect_fnof a urlmatch config, the per-URL form works with no further changes:git config http.https://example.com/.sslVerifyStatus true
It defaults to false, and has to. The option is fail-closed: libcurl fails verification when the server staples nothing at all, so turning this on globally would break every remote that does not staple.
Quoted 6 lines
Leaving the default to libcurl is not an option either. The same complaint was raised there in https://github.com/curl/curl/issues/15483
and closed as intentional ("Marked as enhancement since this was done on purpose"), with the observation that stapling is expected to see less use as Let's Encrypt drops OCSP support. If the check is to be reachable at all, the lever has to come from the application.But... don't we still leave the default to libcurl? If "http.sslVerifyStatus" is not set then we don't touch
CURLOPT_SSL_VERIFYSTATUS, either.
Quoted 5 lines
I might be misreading this though, as the whole commit message is quite hard to digest. I'd assume that this is because it's generated by AI, and it added a lot of the usual weird phrases to the message. It might be a good idea to adapt the message to have a bit more of a human touch to it.
I too had trouble figuring out what the proposed log message really wanted to say, but I wrote it off, blaming the difficulty on a language barrier. But as you said, perhaps it is because it was written by something that does not truly understand what it is talking about. It may not have to explain things to readers as if they were 5 years old, but it is definitely necessary to explain well to readers as if they were humans with average intelligence ;-).
PATCH v5 1/1 http: add http.sslVerifyStatus to check stapled OCSP responses
In reply to Junio C HamanoFrom: Grayson Gordon <[email protected]>
git never sets CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.
Add an http.sslVerifyStatus boolean that maps to CURLOPT_SSL_VERIFYSTATUS.http_options() is already the collect_fn for a urlmatch config, so the per-URL form works with no changes:
git config http.https://example.com/.sslVerifyStatus true
Defaults to false/"off". This is due to the nature of the OCSP protocol. If enabled, git would expect to receive OCSP stapled responses. If the stapled responses were not present, the connection would be blocked as the status of the server's certificate could not be verified. This would break connections to legitimate services that don't use OCSP as their certificate revocation mechanism.
If the backend can't check the staple, curl_easy_setopt() returns CURLE_NOT_BUILT_IN. Error message includes curl_easy_strerror() with the option name to enable users to more easily identify a libcurl built without status verification.
CURLOPT_SSL_VERIFYSTATUS has existed since libcurl 7.41.0, below our 7.61.0 floor, so no version guard is needed.
Tests are in t5551.
--- Documentation/config/http.adoc | 9 +++++++++ http.c | 14 ++++++++++++++ t/t5551-http-fetch-smart.sh | 29 +++++++++++++++++++++++++++++ 3 files changed, 52 insertions(+)
Documentation/config/http.adoc +9 -0
@@ -196,6 +196,15 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verification +fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. + http.sslCert:: File containing the SSL certificate when fetching or pushing over HTTPS. Can be overridden by the `GIT_SSL_CERT` environment
http.c +14 -0
@@ -44,6 +44,7 @@ static CURL *curl_default;char curl_errorstr[CURL_ERROR_SIZE ];static int curl_ssl_verify =-1 ; +static int curl_ssl_verify_status;static int curl_ssl_try;static char *curl_http_version;static char *ssl_cert;
@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))
@@ -1133,6 +1138,15 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +if (curl_ssl_verify_status) { +CURLcode ret =curl_easy_setopt (result, +CURLOPT_SSL_VERIFYSTATUS ,1L ); +if (ret !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but could not " +"enable OCSP status verification: %s" ), +curl_easy_strerror (ret)); + } +if (curl_http_version) {long opt;if (!get_curl_http_version_opt (curl_http_version, &opt)) {
t/t5551-http-fetch-smart.sh +29 -0
@@ -680,6 +680,35 @@ test_expect_success 'passing hostname resolution information works' ' git -c "http.curloptResolve=$BOGUS_HOST:$LIB_HTTPD_PORT:127.0.0.1" ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null ' +test_lazy_prereq SSL_VERIFYSTATUS ' + test "$HTTPD_PROTO" = "https" && + test_might_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" 2>err && + ! grep "http.sslVerifyStatus is set" err +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=true fails without a staple' ' + test_must_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=false is a no-op' ' + git -c http.sslVerifyStatus=false \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus applies to a matching URL' ' + test_must_fail git -c "http.$HTTPD_URL/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus is not applied to other URLs' ' + git -c "http.https://example.com/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + # here user%40host is the URL-encoded version of user@host, # which is our intentionally-odd username to catch parsing errors url_user=$HTTPD_URL_USER/auth/smart/repo.git
Signature · 1 line
2.50.1 (Apple Git-155)
Documentation/config/http.adoc http.sslVerify:: · line 206
Quoted 8 lines · graysongordon-gl
Documentation/config/http.adoc +8
-0+http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verification +fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally.
I do not see us describe a knob or setting that can stop the operation depending on some condition as "fail-closed". Can we rephrase this for regular human beings? Perhaps
Whether to refuse connecting to the server when its certificate has been revoked. Default to false, allowing connection even when its certificate is not known to be still valid.
or something like that might be a good starting point. After all, the "check revocation and/or validity" is *not* the primary objective from the end-user's point of view. Ensuring that they do not talk to suspicious servers is.
Thanks.
Junio,
By "fail-closed" I was specifically referring to the case where no OCSP stapled response was provided. Failing open in this context would mean that, despite verifystatus being set, a response with no stapled response is ALLOWED.
Maybe I'm just a very irregular human being lol.
I'll update the adoc to be similar to what you provided. I refer to the edge cases/different behaviors that we talked about earlier in this thread with the older curl and gnutls versions in the commit message and keep the adoc as simple as possible.
- Grayson
Quoted 27 lines · Junio C Hamano
Documentation/config/http.adoc http.sslVerify:: · line 206
Quoted 8 lines · graysongordon-gl
Documentation/config/http.adoc +8
-0+http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false. ++ +This is fail-closed: if the server staples no response, verification +fails. Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally.I do not see us describe a knob or setting that can stop the operation depending on some condition as "fail-closed". Can we rephrase this for regular human beings? Perhaps
Whether to refuse connecting to the server when its certificate has been revoked. Default to false, allowing connection even when its certificate is not known to be still valid.
or something like that might be a good starting point. After all, the "check revocation and/or validity" is *not* the primary objective from the end-user's point of view. Ensuring that they do not talk to suspicious servers is.
Thanks.
PATCH v6 1/1 http: add http.sslVerifyStatus to check stapled OCSP responses
In reply to Junio C HamanoFrom: Grayson Gordon <[email protected]>
git never sets CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.
Add an http.sslVerifyStatus boolean that maps to CURLOPT_SSL_VERIFYSTATUS.http_options() is already the collect_fn for a urlmatch config, so the per-URL form works with no changes:
git config http.https://example.com/.sslVerifyStatus true
Defaults to false/"off". This is due to the nature of the OCSP protocol. If enabled, git would expect to receive OCSP stapled responses. If the stapled responses were not present, the connection would be blocked as the status of the server's certificate could not be verified. This would break connections to legitimate services that don't use OCSP as their certificate revocation mechanism.
If the backend can't check the staple, curl_easy_setopt() returns CURLE_NOT_BUILT_IN. Error message includes curl_easy_strerror() with the option name to enable users to more easily identify a libcurl built without status verification.
CURLOPT_SSL_VERIFYSTATUS has existed since libcurl 7.41.0, below our 7.61.0 floor, so no version guard is needed.
Tests are in t5551.
Additional note - I put this in http.adoc:
"Defaults to false, which
allows connections to remotes without validating whether or not the certificate has been revoked by the certificate authority."
Technically, there are cases with older combinations of GnuTLS and curl where the revocation logic actually WILL NOT allow such connections. Search "OCSP" in the lore for full details.
--- Documentation/config/http.adoc | 14 ++++++++++++++ http.c | 14 ++++++++++++++ t/t5551-http-fetch-smart.sh | 29 +++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+)
Documentation/config/http.adoc +14 -0
@@ -196,6 +196,20 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false, which + allows connections to servers without validating if the + certificate has been revoked by the certificate authority. + Enabling this option will prevent connections to servers that + have a certificate status other than "good" per RFC 6960. + Connections to servers that do not return a stapled response + will also be refused. ++ +Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. + http.sslCert:: File containing the SSL certificate when fetching or pushing over HTTPS. Can be overridden by the `GIT_SSL_CERT` environment
http.c +14 -0
@@ -44,6 +44,7 @@ static CURL *curl_default;char curl_errorstr[CURL_ERROR_SIZE ];static int curl_ssl_verify =-1 ; +static int curl_ssl_verify_status;static int curl_ssl_try;static char *curl_http_version;static char *ssl_cert;
@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))
@@ -1133,6 +1138,15 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +if (curl_ssl_verify_status) { +CURLcode ret =curl_easy_setopt (result, +CURLOPT_SSL_VERIFYSTATUS ,1L ); +if (ret !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but could not " +"enable OCSP status verification: %s" ), +curl_easy_strerror (ret)); + } +if (curl_http_version) {long opt;if (!get_curl_http_version_opt (curl_http_version, &opt)) {
t/t5551-http-fetch-smart.sh +29 -0
@@ -680,6 +680,35 @@ test_expect_success 'passing hostname resolution information works' ' git -c "http.curloptResolve=$BOGUS_HOST:$LIB_HTTPD_PORT:127.0.0.1" ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null ' +test_lazy_prereq SSL_VERIFYSTATUS ' + test "$HTTPD_PROTO" = "https" && + test_might_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" 2>err && + ! grep "http.sslVerifyStatus is set" err +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=true fails without a staple' ' + test_must_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=false is a no-op' ' + git -c http.sslVerifyStatus=false \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus applies to a matching URL' ' + test_must_fail git -c "http.$HTTPD_URL/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus is not applied to other URLs' ' + git -c "http.https://example.com/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + # here user%40host is the URL-encoded version of user@host, # which is our intentionally-odd username to catch parsing errors url_user=$HTTPD_URL_USER/auth/smart/repo.git
Signature · 1 line
2.50.1 (Apple Git-155)
Hi,
Quoted 1 line · Grayson Gordon
Patrick,
one hint: we prefer to not top-post on this mailing list and instead answer inline.
[snip]
Quoted 12 lines
ON GNUTLS VS OPENSSL DIFFERENCES
I appreciate you including the extra context around GnuTLS 3.8's
GNUTLS_NO_STATUS_REQUESTflag, and curl 8.10 setting it when "verifystatus" is false.Fair enough, said another way, if either of these are true:
Condition 1: curl is being built with GnuTLS version < 3.8. (There's noNO_STATUS_REQUESTflag to set.)
OR
Condition 2: curl version < 8.10. (Curl's not using the flag.)
Quoted 8 lines
You'll see a discrepancy in cert verification behavior between the versions of git built with GnuTLS vs OpenSSL.
While I appreciate the increased precision here, the purpose of my contribution was to expose the functionality that enables git users to set it if they so choose. As it stands today, this option is not presented. So while the discrepancy is what incited me to look deeper, it isn't the central reason I'm here.
That's fair, and I think adding support for OCSP is useful indeed. I just want us to be more accurate in both the commit message and in the docs, as the way is currently written is only partially true and thus misleading both for developers and for readers of git-config(1).
[snip]
Quoted 22 lines
ON COMPREHENSIVE TESTING
I'll leave this at you and Junio's discretion. I worked with him earlier in this thread to avoid introducing another test file and keep the testing succinct.
Right now these tests are just limited to parsing the config and applying it to the user-provided remote.
We COULD do the full suite of tests that cover the full range of cases/behaviors:
- The flag is set AND
- no staple sent (should fail)
- good staple (should pass)
- bad staple (should fail)
etc.We're going to need a lot of infrax for that though:
- test CA.
- test server certificate issued by that CA.
- OCSP responder which knows the certificate's status.
- a way for the TLS server to obtain and staple that response.
- a way to control the response so you can test good vs revoked/invalid.
Quoted 2 lines
I set all of that stuff up in my own experiment repo, emulating this with nginx in docker...
Yeah, it's certainly non-trivial to set all of this up as there's a bunch of pieces to it. Something like the below patch would do it, but I'm not a 100% sure whether it's really worth it given the complexity.
Patrick
t/lib-httpd.sh +119 -5
@@ -25,6 +25,9 @@ # LIB_HTTPD_DAV enable DAV # LIB_HTTPD_SVN enable SVN at given location (e.g. "svn") # LIB_HTTPD_SSL enable SSL +# LIB_HTTPD_OCSP enable OCSP stapling (implies SSL); requires +# the openssl(1) command and needs the caller +# to run start_ocsp_responder after start_httpd # LIB_HTTPD_PROXY enable proxy # # Copyright (c) 2008 Clemens Buchacher <[email protected]>
@@ -171,15 +174,26 @@ prepare_httpd() { ln -s "$LIB_HTTPD_MODULE_PATH" "$HTTPD_ROOT_PATH/modules" + if test -n "$LIB_HTTPD_OCSP" + then + LIB_HTTPD_SSL=t + fi + if test -n "$LIB_HTTPD_SSL" then HTTPD_PROTO=https - RANDFILE_PATH="$HTTPD_ROOT_PATH"/.rnd openssl req \ - -config "$TEST_PATH/ssl.cnf" \ - -new -x509 -nodes \ - -out "$HTTPD_ROOT_PATH/httpd.pem" \ - -keyout "$HTTPD_ROOT_PATH/httpd.pem" + if test -n "$LIB_HTTPD_OCSP" + then + prepare_ocsp_stapling + HTTPD_PARA="$HTTPD_PARA -DOCSP" + else + RANDFILE_PATH="$HTTPD_ROOT_PATH"/.rnd openssl req \ + -config "$TEST_PATH/ssl.cnf" \ + -new -x509 -nodes \ + -out "$HTTPD_ROOT_PATH/httpd.pem" \ + -keyout "$HTTPD_ROOT_PATH/httpd.pem" + fi GIT_SSL_NO_VERIFY=t export GIT_SSL_NO_VERIFY HTTPD_PARA="$HTTPD_PARA -DSSL"
@@ -250,6 +264,106 @@ stop_httpd() { -f "$TEST_PATH/apache.conf" $HTTPD_PARA -k stop } +restart_httpd () { + httpd_pid=$(cat "$HTTPD_ROOT_PATH/httpd.pid") && + stop_httpd && + while kill -0 "$httpd_pid" 2>/dev/null + do + sleep 1 + done && + "$LIB_HTTPD_PATH" -d "$HTTPD_ROOT_PATH" \ + -f "$TEST_PATH/apache.conf" $HTTPD_PARA \ + -c "Listen 127.0.0.1:$LIB_HTTPD_PORT" -k start +} + +# Set up a certificate authority whose certificate httpd.pem is signed +# with, such that "openssl ocsp" can vouch for (or revoke) it. Used +# instead of the self-signed certificate when LIB_HTTPD_OCSP is set. +prepare_ocsp_stapling () { + LIB_HTTPD_OCSP_PORT=$((LIB_HTTPD_PORT + 10000)) + + # Referenced by ocsp-ca.cnf. + OCSP_CA_DIR="$HTTPD_ROOT_PATH/ocsp-ca" + OCSP_URI="http://127.0.0.1:$LIB_HTTPD_OCSP_PORT" + export OCSP_CA_DIR OCSP_URI + + mkdir -p "$OCSP_CA_DIR/newcerts" && + >"$OCSP_CA_DIR/index.txt" && + echo 1000 >"$OCSP_CA_DIR/serial" && + + openssl req -config "$TEST_PATH/ocsp-ca.cnf" \ + -new -x509 -nodes -days 2 \ + -subj "/CN=git-test-ca" -extensions v3_ca \ + -keyout "$HTTPD_ROOT_PATH/ca.key" \ + -out "$HTTPD_ROOT_PATH/ca.pem" && + openssl req -config "$TEST_PATH/ocsp-ca.cnf" \ + -new -nodes \ + -subj "/CN=127.0.0.1" \ + -keyout "$HTTPD_ROOT_PATH/httpd.key" \ + -out "$HTTPD_ROOT_PATH/httpd.csr" && + openssl ca -config "$TEST_PATH/ocsp-ca.cnf" -batch \ + -cert "$HTTPD_ROOT_PATH/ca.pem" \ + -keyfile "$HTTPD_ROOT_PATH/ca.key" \ + -in "$HTTPD_ROOT_PATH/httpd.csr" \ + -out "$HTTPD_ROOT_PATH/httpd.crt" && + cat "$HTTPD_ROOT_PATH/httpd.key" "$HTTPD_ROOT_PATH/httpd.crt" \ + >"$HTTPD_ROOT_PATH/httpd.pem" +} + +run_ocsp_responder () { + openssl ocsp -port "$LIB_HTTPD_OCSP_PORT" \ + -index "$OCSP_CA_DIR/index.txt" \ + -CA "$HTTPD_ROOT_PATH/ca.pem" \ + -rsigner "$HTTPD_ROOT_PATH/ca.pem" \ + -rkey "$HTTPD_ROOT_PATH/ca.key" \ + -nmin 60 >>"$HTTPD_ROOT_PATH/ocsp.log" 2>&1 & + echo $! >"$HTTPD_ROOT_PATH/ocsp.pid" + + for i in $(test_seq 1 10) + do + if openssl ocsp -no_nonce \ + -CAfile "$HTTPD_ROOT_PATH/ca.pem" \ + -issuer "$HTTPD_ROOT_PATH/ca.pem" \ + -cert "$HTTPD_ROOT_PATH/httpd.crt" \ + -url "$OCSP_URI" >/dev/null 2>&1 + then + return 0 + fi + sleep 1 + done + return 1 +} + +start_ocsp_responder () { + test_atexit stop_ocsp_responder + + if ! run_ocsp_responder + then + cat "$HTTPD_ROOT_PATH"/ocsp.log >&4 2>/dev/null + test_skip_or_die GIT_TEST_HTTPD "OCSP responder setup failed" + fi +} + +stop_ocsp_responder () { + if test -f "$HTTPD_ROOT_PATH/ocsp.pid" + then + kill "$(cat "$HTTPD_ROOT_PATH/ocsp.pid")" 2>/dev/null + rm -f "$HTTPD_ROOT_PATH/ocsp.pid" + fi +} + +# Revoke the certificate used by httpd and make both the OCSP responder +# and httpd aware of it. +revoke_httpd_cert () { + openssl ca -config "$TEST_PATH/ocsp-ca.cnf" \ + -cert "$HTTPD_ROOT_PATH/ca.pem" \ + -keyfile "$HTTPD_ROOT_PATH/ca.key" \ + -revoke "$HTTPD_ROOT_PATH/httpd.crt" && + stop_ocsp_responder && + run_ocsp_responder && + restart_httpd +} + test_http_push_nonff () { REMOTE_REPO=$1 LOCAL_REPO=$2
t/lib-httpd/apache.conf +13 -0
@@ -242,6 +242,19 @@ SSLSessionCache none SSLEngine On </IfDefine> +<IfDefine OCSP> +<IfModule !mod_socache_shmcb.c> + LoadModule socache_shmcb_module modules/mod_socache_shmcb.so +</IfModule> + +SSLCertificateChainFile ca.pem +SSLUseStapling On +SSLStaplingCache shmcb:ssl_stapling(65536) +# Also staple responses whose certificate status is not "good", so +# that clients get to see e.g. "revoked" responses. +SSLStaplingReturnResponderErrors On +</IfDefine> + <Location /auth/> AuthType Basic AuthName "git-auth"
t/lib-httpd/ocsp-ca.cnf new file +35 -0
@@ -0,0 +1,35 @@ +[ ca ] +default_ca = CA_default + +[ CA_default ] +dir = $ENV::OCSP_CA_DIR +database = $dir/index.txt +new_certs_dir = $dir/newcerts +serial = $dir/serial +default_md = sha256 +default_days = 2 +policy = policy_anything +email_in_dn = no +unique_subject = no +x509_extensions = server_cert + +[ policy_anything ] +commonName = supplied + +[ req ] +default_bits = 2048 +distinguished_name = req_distinguished_name +prompt = no + +[ req_distinguished_name ] +# The subject is always given on the command line via -subj. + +[ v3_ca ] +basicConstraints = critical, CA:TRUE +keyUsage = critical, digitalSignature, keyCertSign, cRLSign +subjectKeyIdentifier = hash + +[ server_cert ] +basicConstraints = CA:FALSE +subjectAltName = IP:127.0.0.1 +authorityInfoAccess = OCSP;URI:$ENV::OCSP_URI
t/meson.build +1 -0
@@ -727,6 +727,7 @@ integration_tests = [ 't5582-fetch-negative-refspec.sh', 't5583-push-branches.sh', 't5584-http-429-retry.sh', + 't5585-http-ssl-ocsp.sh', 't5600-clone-fail-cleanup.sh', 't5601-clone.sh', 't5602-clone-remote-exec.sh',
t/t5585-http-ssl-ocsp.sh new file +55 -0
55 lines not on this page · raw message
Quoted 40 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.Add an http.sslVerifyStatus boolean that maps to
CURLOPT_SSL_VERIFYSTATUS.http_options()is already thecollect_fnfor a urlmatch config, so the per-URL form works with no changes:git config http.https://example.com/.sslVerifyStatus true
Defaults to false/"off". This is due to the nature of the OCSP protocol. If enabled, git would expect to receive OCSP stapled responses. If the stapled responses were not present, the connection would be blocked as the status of the server's certificate could not be verified. This would break connections to legitimate services that don't use OCSP as their certificate revocation mechanism.
If the backend can't check the staple,
curl_easy_setopt()returnsCURLE_NOT_BUILT_IN. Error message includescurl_easy_strerror()with the option name to enable users to more easily identify a libcurl built without status verification.
CURLOPT_SSL_VERIFYSTATUShas existed since libcurl 7.41.0, below our 7.61.0 floor, so no version guard is needed.Tests are in t5551.
Additional note - I put this in http.adoc:
"Defaults to false, which
allows connections to remotes without validating whether or not the certificate has been revoked by the certificate authority."Technically, there are cases with older combinations of GnuTLS and curl where the revocation logic actually WILL NOT allow such connections. Search "OCSP" in the lore for full details.
Quoted 6 lines
Signed-off-by Grayson Gordon <[email protected]>--- Documentation/config/http.adoc | 14 ++++++++++++++ http.c | 14 ++++++++++++++ t/t5551-http-fetch-smart.sh | 29 +++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+)
Are folks happy with this iteration? I think we have already reached the point of diminishing returns before the thread went dark.
Thanks.
Junio,
Yes, I was hoping for clarity on how thorough we wanted the testing to be. Patrick added a lot of great stuff that I’m happy to use if that’s your preference, but we also talked about wanting to keep the tests succinct. Please let me know what you feel is most appropriate.
- Grayson
Quoted 55 lines · Junio C Hamano
Quoted 40 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.Add an http.sslVerifyStatus boolean that maps to
CURLOPT_SSL_VERIFYSTATUS.http_options()is already thecollect_fnfor a urlmatch config, so the per-URL form works with no changes:git config http.https://example.com/.sslVerifyStatus true
Defaults to false/"off". This is due to the nature of the OCSP protocol. If enabled, git would expect to receive OCSP stapled responses. If the stapled responses were not present, the connection would be blocked as the status of the server's certificate could not be verified. This would break connections to legitimate services that don't use OCSP as their certificate revocation mechanism.
If the backend can't check the staple,
curl_easy_setopt()returnsCURLE_NOT_BUILT_IN. Error message includescurl_easy_strerror()with the option name to enable users to more easily identify a libcurl built without status verification.
CURLOPT_SSL_VERIFYSTATUShas existed since libcurl 7.41.0, below our 7.61.0 floor, so no version guard is needed.Tests are in t5551.
Additional note - I put this in http.adoc:
"Defaults to false, which
allows connections to remotes without validating whether or not the certificate has been revoked by the certificate authority."Technically, there are cases with older combinations of GnuTLS and curl where the revocation logic actually WILL NOT allow such connections. Search "OCSP" in the lore for full details.
Quoted 6 lines
Signed-off-by Grayson Gordon <[email protected]>--- Documentation/config/http.adoc | 14 ++++++++++++++ http.c | 14 ++++++++++++++ t/t5551-http-fetch-smart.sh | 29 +++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+)Are folks happy with this iteration? I think we have already reached the point of diminishing returns before the thread went dark.
Thanks.
Quoted 6 lines · Grayson Gordon
Junio,
Yes, I was hoping for clarity on how thorough we wanted the testing to be. Patrick added a lot of great stuff that I’m happy to use if that’s your preference, but we also talked about wanting to keep the tests succinct. Please let me know what you feel is most appropriate.
If you can keep them succinct but still test the essential bits, that would be great, but I am not sure if that is a great question to ask me ;-) Patrick? You said "not 100% sure given the complexity", but which parts make you feel iffy? They do look involved but seem to cover the situations we do care about, except we seem not to test when the server does not explicitly say "this is still good", or am I not reading the tests correctly?
Thanks.
Quoted 9 lines · Junio C Hamano
Quoted 6 lines · Grayson Gordon
Junio,
Yes, I was hoping for clarity on how thorough we wanted the testing to be. Patrick added a lot of great stuff that I’m happy to use if that’s your preference, but we also talked about wanting to keep the tests succinct. Please let me know what you feel is most appropriate.
Quoted 4 lines
If you can keep them succinct but still test the essential bits, that would be great, but I am not sure if that is a great question to ask me ;-) Patrick? You said "not 100% sure given the complexity", but which parts make you feel iffy?
Setting up OCSP is quite a pain, and that is what made me feel iffy. That being said, given that this is a security-focussed feature I feel like we should probably bite the bullet and verify that we indeed know to reject servers that respond with invalid stapled responses.
And given that this whole setup is now getting more complex I feel like it's worth it to also allocate a new test number for it.
Quoted 3 lines
They do look involved but seem to cover the situations we do care about, except we seem not to test when the server does not explicitly say "this is still good", or am I not reading the tests correctly?
Isn't the following test covering that scenario? Or am I misreading?
test_expect_success SSL_VERIFYSTATUS 'fetch succeeds with stapled "good" OCSP response'
with_ssl_verification git -c http.sslVerifyStatus=true \
ls-remote "$HTTPD_URL/smart/repo.git" >actual &&
test_line_count -gt 0 actual
'Thanks!
Patrick
Quoted 11 lines · Patrick Steinhardt
Quoted 3 lines
They do look involved but seem to cover the situations we do care about, except we seem not to test when the server does not explicitly say "this is still good", or am I not reading the tests correctly?
Isn't the following test covering that scenario? Or am I misreading?
test_expect_success SSL_VERIFYSTATUS 'fetch succeeds with stapled "good" OCSP response' with_ssl_verification git -c http.sslVerifyStatus=true \ ls-remote "$HTTPD_URL/smart/repo.git" >actual && test_line_count -gt 0 actual '
Probably I misstated. What I meant was a reaction to "fail close" floated earlier. A server does not explicitly give stapled good, and the client says "this is not known-good" and not talking to it. I.e. 'fetch fails without stapled "good"'
Quoted 14 lines · Junio C Hamano
Quoted 11 lines · Patrick Steinhardt
Quoted 3 lines
They do look involved but seem to cover the situations we do care about, except we seem not to test when the server does not explicitly say "this is still good", or am I not reading the tests correctly?
Isn't the following test covering that scenario? Or am I misreading?
test_expect_success SSL_VERIFYSTATUS 'fetch succeeds with stapled "good" OCSP response' with_ssl_verification git -c http.sslVerifyStatus=true \ ls-remote "$HTTPD_URL/smart/repo.git" >actual && test_line_count -gt 0 actual '
Quoted 4 lines
Probably I misstated. What I meant was a reaction to "fail close" floated earlier. A server does not explicitly give stapled good, and the client says "this is not known-good" and not talking to it. I.e. 'fetch fails without stapled "good"'
Ah, I think you're correct, my tests didn't include that. But Grayson's already did as it doesn't require any setup, so that's why I didn't include it specifically.
Patrick
Quoted 15 lines · Patrick Steinhardt
Quoted 14 lines · Junio C Hamano
Quoted 11 lines · Patrick Steinhardt
Quoted 3 lines
They do look involved but seem to cover the situations we do care about, except we seem not to test when the server does not explicitly say "this is still good", or am I not reading the tests correctly?
Isn't the following test covering that scenario? Or am I misreading?
test_expect_success SSL_VERIFYSTATUS 'fetch succeeds with stapled "good" OCSP response' with_ssl_verification git -c http.sslVerifyStatus=true \ ls-remote "$HTTPD_URL/smart/repo.git" >actual && test_line_count -gt 0 actual '
Quoted 8 lines
Quoted 4 lines
Probably I misstated. What I meant was a reaction to "fail close" floated earlier. A server does not explicitly give stapled good, and the client says "this is not known-good" and not talking to it. I.e. 'fetch fails without stapled "good"'
Ah, I think you're correct, my tests didn't include that. But Grayson's already did as it doesn't require any setup, so that's why I didn't include it specifically.
Ah, I missed that. So a combined patch taking the best parts from both sides is what we want. Thanks for helping move the topic forward.
hello all,
Sorry I've been away awhile. I'll put together the combined patch and shoot it over.
- Grayson
Quoted 30 lines · Junio C Hamano
Quoted 15 lines · Patrick Steinhardt
Quoted 14 lines · Junio C Hamano
Quoted 11 lines · Patrick Steinhardt
Quoted 3 lines
They do look involved but seem to cover the situations we do care about, except we seem not to test when the server does not explicitly say "this is still good", or am I not reading the tests correctly?
Isn't the following test covering that scenario? Or am I misreading?
test_expect_success SSL_VERIFYSTATUS 'fetch succeeds with stapled "good" OCSP response' with_ssl_verification git -c http.sslVerifyStatus=true \ ls-remote "$HTTPD_URL/smart/repo.git" >actual && test_line_count -gt 0 actual 'Quoted 8 lines
Quoted 4 lines
Probably I misstated. What I meant was a reaction to "fail close" floated earlier. A server does not explicitly give stapled good, and the client says "this is not known-good" and not talking to it. I.e. 'fetch fails without stapled "good"'
Ah, I think you're correct, my tests didn't include that. But Grayson's already did as it doesn't require any setup, so that's why I didn't include it specifically.
Ah, I missed that. So a combined patch taking the best parts from both sides is what we want. Thanks for helping move the topic forward.
PATCH v7 1/1 http: add http.sslVerifyStatus to check stapled OCSP responses
In reply to Junio C HamanoFrom: Grayson Gordon <[email protected]>
git never sets CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.
Add an http.sslVerifyStatus boolean that maps to CURLOPT_SSL_VERIFYSTATUS. http_options() is already the collect_fn for a urlmatch config, so the per-URL form works with no changes:
git config http.https://example.com/.sslVerifyStatus true
Defaults to false/"off". This is due to the nature of the OCSP protocol. If enabled, git would expect to receive OCSP stapled responses. If the stapled responses were not present, the connection would be blocked as the status of the server's certificate could not be verified. This would break connections to legitimate services that don't use OCSP as their certificate revocation mechanism.
If the backend can't check the staple, curl_easy_setopt() returns CURLE_NOT_BUILT_IN. The error message includes curl_easy_strerror() along with the option name, so a libcurl built without status verification is easy to identify.
CURLOPT_SSL_VERIFYSTATUS has existed since libcurl 7.41.0, below our 7.61.0 floor, so no version guard is needed.
The tests that need no OCSP infrastructure stay in t5551, which t5559 runs over https. The rest need a certificate authority, a responder to answer for it and a server configured to staple, so lib-httpd gains an opt-in LIB_HTTPD_OCSP mode and t5585 uses it to check that a "good" staple is accepted, a "revoked" one is refused, and that the revoked one is ignored when the option is off.
---
Junio, Patrick: this is the combined version we discussed. The cases that need no OCSP setup stayed in t5551, since t5559 already runs that file over https, and everything that needs a responder is in the new t5585.
A note on the testing stuff. SSLUseStapling makes apache create a mutex in a compiled-in system-wide runtime directory. I set DefaultRuntimeDir in the OCSP block to keep that mutex in the server root, the other way resolved to a path on my box that didn't exist and prevented the server from starting.
Changes since v6:
- added t5585 and LIB_HTTPD_OCSP support in lib-httpd, taken from Patrick's patch
- moved the SSL_VERIFYSTATUS prereq into lib-httpd.sh so both files share one definition
Documentation/config/http.adoc | 14 ++++ http.c | 14 ++++ t/lib-httpd.sh | 130 +++++++++++++++++++++++++++++++-- t/lib-httpd/apache.conf | 16 ++++ t/lib-httpd/ocsp-ca.cnf | 35 +++++++++ t/meson.build | 1 + t/t5551-http-fetch-smart.sh | 22 ++++++ t/t5585-http-ssl-ocsp.sh | 55 ++++++++++++++ 8 files changed, 282 insertions(+), 5 deletions(-)
Documentation/config/http.adoc +14 -0
@@ -196,6 +196,20 @@ http.sslVerify:: over HTTPS. Defaults to true. Can be overridden by the `GIT_SSL_NO_VERIFY` environment variable. +http.sslVerifyStatus:: + Whether to check the revocation status of the server + certificate using the stapled OCSP response supplied during + the TLS handshake ("OCSP stapling"). Defaults to false, which + allows connections to servers without validating if the + certificate has been revoked by the certificate authority. + Enabling this option will prevent connections to servers that + have a certificate status other than "good" per RFC 6960. + Connections to servers that do not return a stapled response + will also be refused. ++ +Set it per remote, e.g. +`http.https://example.com/.sslVerifyStatus`, rather than globally. + http.sslCert:: File containing the SSL certificate when fetching or pushing over HTTPS. Can be overridden by the `GIT_SSL_CERT` environment
http.c +14 -0
@@ -44,6 +44,7 @@ static CURL *curl_default;char curl_errorstr[CURL_ERROR_SIZE ];static int curl_ssl_verify =-1 ; +static int curl_ssl_verify_status;static int curl_ssl_try;static char *curl_http_version;static char *ssl_cert;
@@ -400,6 +401,10 @@ static int http_options(const char *var, const char *value, curl_ssl_verify =git_config_bool (var, value);return 0 ; } +if (!strcmp ("http.sslverifystatus" , var)) { + curl_ssl_verify_status =git_config_bool (var, value); +return 0 ; + }if (!strcmp ("http.sslcipherlist" , var))return git_config_string (&ssl_cipherlist, var, value);if (!strcmp ("http.sslversion" , var))
@@ -1133,6 +1138,15 @@ static CURL *get_curl_handle(void)curl_easy_setopt (result,CURLOPT_SSL_VERIFYHOST ,2L ); } +if (curl_ssl_verify_status) { +CURLcode ret =curl_easy_setopt (result, +CURLOPT_SSL_VERIFYSTATUS ,1L ); +if (ret !=CURLE_OK ) +die (_ ("http.sslVerifyStatus is set, but could not " +"enable OCSP status verification: %s" ), +curl_easy_strerror (ret)); + } +if (curl_http_version) {long opt;if (!get_curl_http_version_opt (curl_http_version, &opt)) {
t/lib-httpd.sh +125 -5
152 lines not on this page · raw message
t/lib-httpd/apache.conf +16 -0
@@ -242,6 +242,22 @@ SSLSessionCache none SSLEngine On </IfDefine> +<IfDefine OCSP> +<IfModule !mod_socache_shmcb.c> + LoadModule socache_shmcb_module modules/mod_socache_shmcb.so +</IfModule> + +SSLCertificateChainFile ca.pem +SSLUseStapling On +# Stapling needs a mutex, which apache would put in a system-wide +# runtime directory that need not be writable. Keep it in the server +# root, or httpd refuses to start instead of skipping the tests. +DefaultRuntimeDir . +SSLStaplingCache shmcb:ssl_stapling(65536) +# Staple non-"good" responses too, so clients get to see "revoked". +SSLStaplingReturnResponderErrors On +</IfDefine> + <Location /auth/> AuthType Basic AuthName "git-auth"
t/lib-httpd/ocsp-ca.cnf new file +35 -0
@@ -0,0 +1,35 @@ +[ ca ] +default_ca = CA_default + +[ CA_default ] +dir = $ENV::OCSP_CA_DIR +database = $dir/index.txt +new_certs_dir = $dir/newcerts +serial = $dir/serial +default_md = sha256 +default_days = 2 +policy = policy_anything +email_in_dn = no +unique_subject = no +x509_extensions = server_cert + +[ policy_anything ] +commonName = supplied + +[ req ] +default_bits = 2048 +distinguished_name = req_distinguished_name +prompt = no + +[ req_distinguished_name ] +# The subject is always given on the command line via -subj. + +[ v3_ca ] +basicConstraints = critical, CA:TRUE +keyUsage = critical, digitalSignature, keyCertSign, cRLSign +subjectKeyIdentifier = hash + +[ server_cert ] +basicConstraints = CA:FALSE +subjectAltName = IP:127.0.0.1 +authorityInfoAccess = OCSP;URI:$ENV::OCSP_URI
t/meson.build +1 -0
@@ -728,6 +728,7 @@ integration_tests = [ 't5582-fetch-negative-refspec.sh', 't5583-push-branches.sh', 't5584-http-429-retry.sh', + 't5585-http-ssl-ocsp.sh', 't5600-clone-fail-cleanup.sh', 't5601-clone.sh', 't5602-clone-remote-exec.sh',
t/t5551-http-fetch-smart.sh +22 -0
@@ -680,6 +680,28 @@ test_expect_success 'passing hostname resolution information works' ' git -c "http.curloptResolve=$BOGUS_HOST:$LIB_HTTPD_PORT:127.0.0.1" ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null ' +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=true fails without a staple' ' + test_must_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=false is a no-op' ' + git -c http.sslVerifyStatus=false \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus applies to a matching URL' ' + test_must_fail git -c "http.$HTTPD_URL/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus is not applied to other URLs' ' + git -c "http.https://example.com/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + # here user%40host is the URL-encoded version of user@host, # which is our intentionally-odd username to catch parsing errors url_user=$HTTPD_URL_USER/auth/smart/repo.git
t/t5585-http-ssl-ocsp.sh new file +55 -0
55 lines not on this page · raw message
Signature · 1 line
2.50.1 (Apple Git-155)
Quoted 23 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.Add an http.sslVerifyStatus boolean that maps to
CURLOPT_SSL_VERIFYSTATUS.http_options()is already thecollect_fnfor a urlmatch config, so the per-URL form works with no changes:
---Junio, Patrick: this is the combined version we discussed. The cases that need no OCSP setup stayed in t5551, since t5559 already runs that file over https, and everything that needs a responder is in the new t5585.
A note on the testing stuff. SSLUseStapling makes apache create a mutex in a compiled-in system-wide runtime directory. I set DefaultRuntimeDir in the OCSP block to keep that mutex in the server root, the other way resolved to a path on my box that didn't exist and prevented the server from starting.
Quoted 5 lines
Changes since v6:
- added t5585 andLIB_HTTPD_OCSPsupport in lib-httpd, taken from Patrick's patch
- moved theSSL_VERIFYSTATUSprereq intolib-httpd.shso both files share one definition
The updated tests look good; will replace. Thanks.
Quoted 20 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.Add an http.sslVerifyStatus boolean that maps to
CURLOPT_SSL_VERIFYSTATUS.http_options()is already thecollect_fnfor a urlmatch config, so the per-URL form works with no changes:git config http.https://example.com/.sslVerifyStatus true
Defaults to false/"off". This is due to the nature of the OCSP protocol. If enabled, git would expect to receive OCSP stapled responses. If the stapled responses were not present, the connection would be blocked as the status of the server's certificate could not be verified. This would break connections to legitimate services that don't use OCSP as their certificate revocation mechanism.
Quoted 4 lines
If the backend can't check the staple,
curl_easy_setopt()returnsCURLE_NOT_BUILT_IN. The error message includescurl_easy_strerror()along with the option name, so a libcurl built without status verification is easy to identify.
Nit: I feel like this paragraph is excessive information, as it doesn't give the reviewer any additional context over what the code already states.
Quoted 9 lines
CURLOPT_SSL_VERIFYSTATUShas existed since libcurl 7.41.0, below our 7.61.0 floor, so no version guard is needed.The tests that need no OCSP infrastructure stay in t5551, which t5559 runs over https. The rest need a certificate authority, a responder to answer for it and a server configured to staple, so lib-httpd gains an opt-in
LIB_HTTPD_OCSPmode and t5585 uses it to check that a "good" staple is accepted, a "revoked" one is refused, and that the revoked one is ignored when the option is off.
Nit: Likewise, this paragraph doesn't add much value.
Other than that I'm happy with this patch. I'll leave it to you (or others) to decide whether this requires another reroll to address the two nits.
Thanks!
Patrick
Quoted 35 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.Add an http.sslVerifyStatus boolean that maps to
CURLOPT_SSL_VERIFYSTATUS.http_options()is already thecollect_fnfor a urlmatch config, so the per-URL form works with no changes:git config http.https://example.com/.sslVerifyStatus true
Defaults to false/"off". This is due to the nature of the OCSP protocol. If enabled, git would expect to receive OCSP stapled responses. If the stapled responses were not present, the connection would be blocked as the status of the server's certificate could not be verified. This would break connections to legitimate services that don't use OCSP as their certificate revocation mechanism.
If the backend can't check the staple,
curl_easy_setopt()returnsCURLE_NOT_BUILT_IN. The error message includescurl_easy_strerror()along with the option name, so a libcurl built without status verification is easy to identify.
CURLOPT_SSL_VERIFYSTATUShas existed since libcurl 7.41.0, below our 7.61.0 floor, so no version guard is needed.The tests that need no OCSP infrastructure stay in t5551, which t5559 runs over https. The rest need a certificate authority, a responder to answer for it and a server configured to staple, so lib-httpd gains an opt-in
LIB_HTTPD_OCSPmode and t5585 uses it to check that a "good" staple is accepted, a "revoked" one is refused, and that the revoked one is ignored when the option is off.
Quoted 2 lines
Signed-off-by Grayson Gordon <[email protected]>---
This patch was merged to 'next' the other day, and the last test in the new t5585 fails on my system.
Quoted 57 lines
t/lib-httpd.sh +26
-5@@ -25,6 +25,7 @@ # LIB_HTTPD_DAV enable DAV # LIB_HTTPD_SVN enable SVN at given location (e.g. "svn") # LIB_HTTPD_SSL enable SSL +# LIB_HTTPD_OCSP enable OCSP stapling # LIB_HTTPD_PROXY enable proxy # # Copyright (c) 2008 Clemens Buchacher <[email protected]>@@ -183,15 +184,26 @@ prepare_httpd() { ln -s "$LIB_HTTPD_MODULE_PATH" "$HTTPD_ROOT_PATH/modules" + if test -n "$LIB_HTTPD_OCSP" + then + LIB_HTTPD_SSL=t + fi + if test -n "$LIB_HTTPD_SSL" then HTTPD_PROTO=https - RANDFILE_PATH="$HTTPD_ROOT_PATH"/.rnd openssl req \ - -config "$TEST_PATH/ssl.cnf" \ - -new -x509 -nodes \ - -out "$HTTPD_ROOT_PATH/httpd.pem" \ - -keyout "$HTTPD_ROOT_PATH/httpd.pem" + if test -n "$LIB_HTTPD_OCSP" + then + prepare_ocsp_stapling + HTTPD_PARA="$HTTPD_PARA -DOCSP" + else + RANDFILE_PATH="$HTTPD_ROOT_PATH"/.rnd openssl req \ + -config "$TEST_PATH/ssl.cnf" \ + -new -x509 -nodes \ + -out "$HTTPD_ROOT_PATH/httpd.pem" \ + -keyout "$HTTPD_ROOT_PATH/httpd.pem" + fi GIT_SSL_NO_VERIFY=t export GIT_SSL_NO_VERIFY HTTPD_PARA="$HTTPD_PARA -DSSL"@@ -262,6 +274,114 @@ stop_httpd() { -f "$TEST_PATH/apache.conf" $HTTPD_PARA -k stop } +restart_httpd () { + httpd_pid=$(cat "$HTTPD_ROOT_PATH/httpd.pid") && + stop_httpd && + while kill -0 "$httpd_pid" 2>/dev/null + do + sleep 1 + done && + "$LIB_HTTPD_PATH" -d "$HTTPD_ROOT_PATH" \ + -f "$TEST_PATH/apache.conf" $HTTPD_PARA \
t/lib-httpd.sh stop_httpd() { · line 295
Quoted 10 lines
t/lib-httpd.sh +10
-0+ -c "Listen 127.0.0.1:$LIB_HTTPD_PORT" -k start +} + +# Check if the linked libcurl can verify stapled OCSP responses. +test_lazy_prereq SSL_VERIFYSTATUS ' + test "$HTTPD_PROTO" = "https" && + test_might_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL" 2>err && + ! grep "http.sslVerifyStatus is set" err +'
When checking this prereq in t5585, I get the following trace:
mkdir -p "$TRASH_DIRECTORY/prereq-test-dir-SSL_VERIFYSTATUS" && ( cd "$TRASH_DIRECTORY/prereq-test-dir-SSL_VERIFYSTATUS" && test "$HTTPD_PROTO" = "https" && test_might_fail git -c http.sslVerifyStatus=true \ ls-remote "$HTTPD_URL" 2>err && cat err && # debug ! grep "http.sslVerifyStatus is set" err
)
+ mkdir -p /home/szeder/src/git/t/trash directory.t5585-http-ssl-ocsp/prereq-test-dir-SSL_VERIFYSTATUS
+ cd /home/szeder/src/git/t/trash directory.t5585-http-ssl-ocsp/prereq-test-dir-SSL_VERIFYSTATUS
+ test https = https
+ test_might_fail git -c http.sslVerifyStatus=true ls-remote https://127.0.0.1:5585
+ cat err
fatal: repository 'https://127.0.0.1:5585/' not found
+ grep http.sslVerifyStatus is set err
prerequisite SSL_VERIFYSTATUS ok
I added that 'cat err' to see the error message. Turns out that 'git ls-remote' can't even find the repository on the remote, but the prereq is still considered fulfilled. Is that right?
In t5559 I get the following trace:
+ mkdir -p /home/szeder/src/git/t/trash directory.t5559-http-fetch-smart-http2/prereq-test-dir-SSL_VERIFYSTATUS
+ cd /home/szeder/src/git/t/trash directory.t5559-http-fetch-smart-http2/prereq-test-dir-SSL_VERIFYSTATUS
+ test https = https
+ test_might_fail git -c http.sslVerifyStatus=true ls-remote https://127.0.0.1:5559
+ cat err
fatal: unable to access 'https://127.0.0.1:5559/': No OCSP response received
+ grep http.sslVerifyStatus is set err
prerequisite SSL_VERIFYSTATUS ok
This time the error message talks about missing OCSP response, but the prereq is still considered fulfilled. Again: is that right?!
Instead of the lack of a certain string in the error message, is there something positive that we can test instead?
Quoted 176 lines
t/lib-httpd.sh +88
-0+# Set up a certificate authority. It issues certificate "httpd.pem" +# and is able to revoke it. Used instead of the self-signed +# certificate when LIB_HTTPD_OCSP is set. +prepare_ocsp_stapling () { + LIB_HTTPD_OCSP_PORT=$((LIB_HTTPD_PORT + 10000)) + + # Referenced by ocsp-ca.cnf. + OCSP_CA_DIR="$HTTPD_ROOT_PATH/ocsp-ca" + OCSP_URI="http://127.0.0.1:$LIB_HTTPD_OCSP_PORT" + export OCSP_CA_DIR OCSP_URI + + mkdir -p "$OCSP_CA_DIR/newcerts" && + >"$OCSP_CA_DIR/index.txt" && + echo 1000 >"$OCSP_CA_DIR/serial" && + + openssl req -config "$TEST_PATH/ocsp-ca.cnf" \ + -new -x509 -nodes -days 2 \ + -subj "/CN=git-test-ca" -extensions v3_ca \ + -keyout "$HTTPD_ROOT_PATH/ca.key" \ + -out "$HTTPD_ROOT_PATH/ca.pem" && + openssl req -config "$TEST_PATH/ocsp-ca.cnf" \ + -new -nodes \ + -subj "/CN=127.0.0.1" \ + -keyout "$HTTPD_ROOT_PATH/httpd.key" \ + -out "$HTTPD_ROOT_PATH/httpd.csr" && + openssl ca -config "$TEST_PATH/ocsp-ca.cnf" -batch \ + -cert "$HTTPD_ROOT_PATH/ca.pem" \ + -keyfile "$HTTPD_ROOT_PATH/ca.key" \ + -in "$HTTPD_ROOT_PATH/httpd.csr" \ + -out "$HTTPD_ROOT_PATH/httpd.crt" && + cat "$HTTPD_ROOT_PATH/httpd.key" "$HTTPD_ROOT_PATH/httpd.crt" \ + >"$HTTPD_ROOT_PATH/httpd.pem" +} + +run_ocsp_responder () { + openssl ocsp -port "$LIB_HTTPD_OCSP_PORT" \ + -index "$OCSP_CA_DIR/index.txt" \ + -CA "$HTTPD_ROOT_PATH/ca.pem" \ + -rsigner "$HTTPD_ROOT_PATH/ca.pem" \ + -rkey "$HTTPD_ROOT_PATH/ca.key" \ + -nmin 60 >>"$HTTPD_ROOT_PATH/ocsp.log" 2>&1 & + echo $! >"$HTTPD_ROOT_PATH/ocsp.pid" + + for i in $(test_seq 1 10) + do + if openssl ocsp -no_nonce \ + -CAfile "$HTTPD_ROOT_PATH/ca.pem" \ + -issuer "$HTTPD_ROOT_PATH/ca.pem" \ + -cert "$HTTPD_ROOT_PATH/httpd.crt" \ + -url "$OCSP_URI" >/dev/null 2>&1 + then + return 0 + fi + sleep 1 + done + return 1 +} + +start_ocsp_responder () { + test_atexit stop_ocsp_responder + + if ! run_ocsp_responder + then + cat "$HTTPD_ROOT_PATH"/ocsp.log >&4 2>/dev/null + test_skip_or_die GIT_TEST_HTTPD "OCSP responder setup failed" + fi +} + +stop_ocsp_responder () { + if test -f "$HTTPD_ROOT_PATH/ocsp.pid" + then + kill "$(cat "$HTTPD_ROOT_PATH/ocsp.pid")" 2>/dev/null + rm -f "$HTTPD_ROOT_PATH/ocsp.pid" + fi +} + +# Revoke the certificate used by httpd and make both the OCSP responder +# and httpd aware of it. +revoke_httpd_cert () { + openssl ca -config "$TEST_PATH/ocsp-ca.cnf" \ + -cert "$HTTPD_ROOT_PATH/ca.pem" \ + -keyfile "$HTTPD_ROOT_PATH/ca.key" \ + -revoke "$HTTPD_ROOT_PATH/httpd.crt" && + stop_ocsp_responder && + run_ocsp_responder && + restart_httpd +} + test_http_push_nonff () { REMOTE_REPO=$1 LOCAL_REPO=$2t/lib-httpd/apache.conf +16
-0@@ -242,6 +242,22 @@ SSLSessionCache none SSLEngine On </IfDefine> +<IfDefine OCSP> +<IfModule !mod_socache_shmcb.c> + LoadModule socache_shmcb_module modules/mod_socache_shmcb.so +</IfModule> + +SSLCertificateChainFile ca.pem +SSLUseStapling On +# Stapling needs a mutex, which apache would put in a system-wide +# runtime directory that need not be writable. Keep it in the server +# root, or httpd refuses to start instead of skipping the tests. +DefaultRuntimeDir . +SSLStaplingCache shmcb:ssl_stapling(65536) +# Staple non-"good" responses too, so clients get to see "revoked". +SSLStaplingReturnResponderErrors On +</IfDefine> + <Location /auth/> AuthType Basic AuthName "git-auth"t/lib-httpd/ocsp-ca.cnf new file +35
-0@@ -0,0 +1,35 @@ +[ ca ] +default_ca = CA_default + +[ CA_default ] +dir = $ENV::OCSP_CA_DIR +database = $dir/index.txt +new_certs_dir = $dir/newcerts +serial = $dir/serial +default_md = sha256 +default_days = 2 +policy = policy_anything +email_in_dn = no +unique_subject = no +x509_extensions = server_cert + +[ policy_anything ] +commonName = supplied + +[ req ] +default_bits = 2048 +distinguished_name = req_distinguished_name +prompt = no + +[ req_distinguished_name ] +# The subject is always given on the command line via -subj. + +[ v3_ca ] +basicConstraints = critical, CA:TRUE +keyUsage = critical, digitalSignature, keyCertSign, cRLSign +subjectKeyIdentifier = hash + +[ server_cert ] +basicConstraints = CA:FALSE +subjectAltName = IP:127.0.0.1 +authorityInfoAccess = OCSP;URI:$ENV::OCSP_URIt/meson.build +1
-0@@ -728,6 +728,7 @@ integration_tests = [ 't5582-fetch-negative-refspec.sh', 't5583-push-branches.sh', 't5584-http-429-retry.sh', + 't5585-http-ssl-ocsp.sh', 't5600-clone-fail-cleanup.sh', 't5601-clone.sh', 't5602-clone-remote-exec.sh',t/t5551-http-fetch-smart.sh +0
-0
t/t5551-http-fetch-smart.sh test_expect_success 'passing hostname resolution information works' ' · line 685
Quoted 6 lines
t/t5551-http-fetch-smart.sh +3
-0@@ -680,6 +680,28 @@ test_expect_success 'passing hostname resolution information works' ' git -c "http.curloptResolve=$BOGUS_HOST:$LIB_HTTPD_PORT:127.0.0.1" ls-remote "$BOGUS_HTTPD_URL/smart/repo.git" >/dev/null ' +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=true fails without a staple' ' + test_must_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git"
Shouldn't we check the error message, to make sure that the command failed for the expected reason (here and in t5585 as well)?
Quoted 45 lines
t/t5551-http-fetch-smart.sh +19
-0+' + +test_expect_success SSL_VERIFYSTATUS 'http.sslVerifyStatus=false is a no-op' ' + git -c http.sslVerifyStatus=false \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus applies to a matching URL' ' + test_must_fail git -c "http.$HTTPD_URL/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" +' + +test_expect_success SSL_VERIFYSTATUS 'per-URL sslVerifyStatus is not applied to other URLs' ' + git -c "http.https://example.com/.sslVerifyStatus=true" \ + ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + # here user%40host is the URL-encoded version of user@host, # which is our intentionally-odd username to catch parsing errors url_user=$HTTPD_URL_USER/auth/smart/repo.gitt/t5585-http-ssl-ocsp.sh new file +17
-0@@ -0,0 +1,55 @@ +#!/bin/sh + +test_description='verification of stapled OCSP responses via http.sslVerifyStatus' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +. ./test-lib.sh + +LIB_HTTPD_OCSP=1 +. "$TEST_DIRECTORY"/lib-httpd.sh + +start_httpd +start_ocsp_responder + +test_expect_success 'setup repository' ' + test_commit one &&
t/t5585-http-ssl-ocsp.sh line 27
Quoted 10 lines
t/t5585-http-ssl-ocsp.sh new file +10
-0+ git init --bare "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" && + git push "$HTTPD_DOCUMENT_ROOT_PATH/repo.git" HEAD:refs/heads/main +' + +# lib-httpd.sh exports GIT_SSL_NO_VERIFY, which would keep us from ever +# looking at the certificate. Trust our own CA instead. +with_ssl_verification () { + ( + sane_unset GIT_SSL_NO_VERIFY && + GIT_SSL_CAINFO="$HTTPD_ROOT_PATH/ca.pem" "$@"
According to our CodingGuidelines, a temporary variable assignment like this should not be used for shell functions for portability reasons. In most test cases this is fine, becase "$@" is a git command, but ...
Quoted 10 lines
t/t5585-http-ssl-ocsp.sh new file +10
-0+ ) +} + +test_expect_success SSL_VERIFYSTATUS 'certificate verification works against test CA' ' + with_ssl_verification git ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + +test_expect_success SSL_VERIFYSTATUS 'fetch succeeds with stapled "good" OCSP response' ' + with_ssl_verification git -c http.sslVerifyStatus=true \
t/t5585-http-ssl-ocsp.sh line 47
Quoted 10 lines
t/t5585-http-ssl-ocsp.sh new file +10
-0+ ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +' + +test_expect_success SSL_VERIFYSTATUS 'revoked certificate is rejected' ' + revoke_httpd_cert && + with_ssl_verification test_must_fail git -c http.sslVerifyStatus=true \ + ls-remote "$HTTPD_URL/smart/repo.git" 2>err && + test_grep -i -e "ocsp" -e "revocation" -e "revoked" -e "certificate status" err +'
... in this case "$@" is the test_must_fail shell function.
Please set and then export that variable instead; it's already in a subshell because of the sane_unset anyway.
t/t5585-http-ssl-ocsp.sh line 53
Quoted 5 lines
t/t5585-http-ssl-ocsp.sh new file +5
-0+# Depends on the certificate revoked by the preceding test. +test_expect_success SSL_VERIFYSTATUS 'revoked certificate is accepted without http.sslVerifyStatus' ' + with_ssl_verification git ls-remote "$HTTPD_URL/smart/repo.git" >actual && + test_line_count -gt 0 actual +'
So this test case fails for me with the following trace output:
expecting success of 5585.5 'revoked certificate is accepted without http.sslVerifyStatus':
with_ssl_verification git ls-remote "$HTTPD_URL/smart/repo.git" >actual && test_line_count -gt 0 actual
+ with_ssl_verification git ls-remote https://127.0.0.1:5585/smart/repo.git
+ sane_unset GIT_SSL_NO_VERIFY
+ unset GIT_SSL_NO_VERIFY
+ return 0
+ GIT_SSL_CAINFO=/home/szeder/src/git/t/trash directory.t5585-http-ssl-ocsp/httpd/ca.pem git ls-remote https://127.0.0.1:5585/smart/repo.git
fatal: unable to access 'https://127.0.0.1:5585/smart/repo.git/': server certificate verification failed. CAfile: /home/szeder/src/git/t/trash directory.t5585-http-ssl-ocsp/httpd/ca.pem CRLfile: none
error: last command exited with $?=128
not ok 5 - revoked certificate is accepted without http.sslVerifyStatus
#
# with_ssl_verification git ls-remote "$HTTPD_URL/smart/repo.git" >actual &&
# test_line_count -gt 0 actual
#
libcurl is 7.81.0, apache is 2.4.52 (whatever is shipped in this slowly aging LTS...)
Quoted 5 lines · Patrick Steinhardt
Nit: Likewise, this paragraph doesn't add much value.
Other than that I'm happy with this patch. I'll leave it to you (or others) to decide whether this requires another reroll to address the two nits.
SZEDER reports breakages with this topic.
https://lore.kernel.org/git/arQ%2FnOH+o3XwQFD%[email protected]/
Since we are not in a hurry to take this topic, let me revert it out of 'next' and give it time to mature. When the reroll comes, we can critique these overly verbose words without much meaning again.
Thanks.
Quoted 10 lines · SZEDER Gábor
Quoted 6 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked....
This patch was merged to 'next' the other day, and the last test in the new t5585 fails on my system.
Sorry about a premature merge. Since we are not in a hurry to take this topic in (or no new feature topic in general), let me revert it out of 'next' and give it a clean slate to try again.
Quoted 10 lines
...
I added that 'cat err' to see the error message. Turns out that 'git ls-remote' can't even find the repository on the remote, but the prereq is still considered fulfilled. Is that right?
...
This time the error message talks about missing OCSP response, but the prereq is still considered fulfilled. Again: is that right?!Instead of the lack of a certain string in the error message, is there something positive that we can test instead?
Oh, that is a very constructive and useful suggestion. Greatly appreciated.
Quoted 19 lines
So this test case fails for me with the following trace output:
expecting success of 5585.5 'revoked certificate is accepted without http.sslVerifyStatus':
with_ssl_verificationgit ls-remote "$HTTPD_URL/smart/repo.git" >actual &&test_line_count-gt 0 actual+
with_ssl_verificationgit ls-remote https://127.0.0.1:5585/smart/repo.git
+sane_unsetGIT_SSL_NO_VERIFY
+ unsetGIT_SSL_NO_VERIFY
+ return 0
+ GIT_SSL_CAINFO=/home/szeder/src/git/t/trash directory.t5585-http-ssl-ocsp/httpd/ca.pem git ls-remote https://127.0.0.1:5585/smart/repo.git
fatal: unable to access 'https://127.0.0.1:5585/smart/repo.git/': server certificate verification failed. CAfile:/home/szeder/src/git/t/trashdirectory.t5585-http-ssl-ocsp/httpd/ca.pem CRLfile: none
error: last command exited with $?=128
not ok 5 - revoked certificate is accepted without http.sslVerifyStatus
#
#with_ssl_verificationgit ls-remote "$HTTPD_URL/smart/repo.git" >actual &&
#test_line_count-gt 0 actual
#
Quoted 2 lines
libcurl is 7.81.0, apache is 2.4.52 (whatever is shipped in this slowly aging LTS...)
Thanks.
Quoted 9 lines · Junio C Hamano
Quoted 7 lines · SZEDER Gábor
Quoted 6 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.
Quoted 7 lines
Quoted 3 lines
...
This patch was merged to 'next' the other day, and the last test in the new t5585 fails on my system.Sorry about a premature merge. Since we are not in a hurry to take this topic in (or no new feature topic in general), let me revert it out of 'next' and give it a clean slate to try again.
Well, if you hadn't merged it, we would perhaps still be none the wiser, because, alas, I don't have the bandwidth to run tests on the seen branch regularly...
However, CI does, but I can't seem to find any CI runs that failed because of this, which makes me worried that something is wrong on my end.
Quoted 8 lines
Quoted 8 lines
...
I added that 'cat err' to see the error message. Turns out that 'git ls-remote' can't even find the repository on the remote, but the prereq is still considered fulfilled. Is that right?
...
This time the error message talks about missing OCSP response, but the prereq is still considered fulfilled. Again: is that right?!
Quoted 5 lines
Quoted 2 lines
Instead of the lack of a certain string in the error message, is there something positive that we can test instead?
Oh, that is a very constructive and useful suggestion. Greatly appreciated.
After having slept on it :) I now start to realize that this SSL_VERIFYSTATUS prereq only checks that libcurl supports the CURLOPT_SSL_VERIFYSTATUS option, and has nothing to do with the capabilities and configuration of the web server. If my understanding is correct, then I think that:
- Merely attempting a connection to somewhere is indeed sufficient to check this, and it doesn't matter that the server can't find the requested repository.
- Checking for the error message printed after curl_easy_setopt(..., CURLOPT_SSL_VERIFYSTATUS, ...) returns with error is indeed the right thing to do.
However, in that new error message the second half is much more informative than the first, and if the prereq looked for the absence of "could not enable OCSP status verification" instead of "http.sslVerifyStatus is set", then I think I would have realized all this sooner. Perhaps calling the prereq CURL_SSL_VERIFYSTATUS would have helped, too.
Quoted 18 lines · SZEDER Gábor
Quoted 9 lines · Junio C Hamano
Quoted 7 lines · SZEDER Gábor
Quoted 6 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.Quoted 7 lines
Quoted 3 lines
...
This patch was merged to 'next' the other day, and the last test in the new t5585 fails on my system.Sorry about a premature merge. Since we are not in a hurry to take this topic in (or no new feature topic in general), let me revert it out of 'next' and give it a clean slate to try again.
Quoted 7 lines
Well, if you hadn't merged it, we would perhaps still be none the wiser, because, alas, I don't have the bandwidth to run tests on the seen branch regularly...
However, CI does, but I can't seem to find any CI runs that failed because of this, which makes me worried that something is wrong on my end.
Do you maybe run with a curl backend that doesn't properly support OCSP? But even if so, our test suite should notice and skip the tests.
Patrick
Quoted 10 lines · SZEDER Gábor
- Checking for the error message printed after
curl_easy_setopt(..., CURLOPT_SSL_VERIFYSTATUS, ...)returns with error is indeed the right thing to do.However, in that new error message the second half is much more informative than the first, and if the prereq looked for the absence of "could not enable OCSP status verification" instead of "http.sslVerifyStatus is set", then I think I would have realized all this sooner. Perhaps calling the prereq
CURL_SSL_VERIFYSTATUSwould have helped, too.
Yeah, both are understandable.
Thanks.
Quoted 23 lines · Patrick Steinhardt
Quoted 22 lines · SZEDER Gábor
Quoted 9 lines · Junio C Hamano
Quoted 7 lines · SZEDER Gábor
Quoted 6 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.Quoted 7 lines
Quoted 3 lines
...
This patch was merged to 'next' the other day, and the last test in the new t5585 fails on my system.Sorry about a premature merge. Since we are not in a hurry to take this topic in (or no new feature topic in general), let me revert it out of 'next' and give it a clean slate to try again.
Well, if you hadn't merged it, we would perhaps still be none the wiser, because, alas, I don't have the bandwidth to run tests on the seen branch regularly...
Quoted 6 lines
Quoted 3 lines
However, CI does, but I can't seem to find any CI runs that failed because of this, which makes me worried that something is wrong on my end.
Do you maybe run with a curl backend that doesn't properly support OCSP? But even if so, our test suite should notice and skip the tests.
Apparently I did! Removing 'libcurl4-gnutls-dev' and installing 'libcurl4-openssl-dev' instead makes t5585 succeed. Go figure.
Thanks for the hint!
Quoted 7 lines · SZEDER Gábor
Quoted 2 lines
Do you maybe run with a curl backend that doesn't properly support OCSP? But even if so, our test suite should notice and skip the tests.
Apparently I did! Removing 'libcurl4-gnutls-dev' and installing 'libcurl4-openssl-dev' instead makes t5585 succeed. Go figure.
Thanks for the hint!
Thanks for collectively digging down the cause of the issue to (1) help your set-up to pass the test, and (2) point out that the prerequisite setting needs to be improved.
Quoted 18 lines · SZEDER Gábor
Quoted 9 lines · Junio C Hamano
Quoted 7 lines · SZEDER Gábor
Quoted 6 lines · graysongordon-gl
From: Grayson Gordon <[email protected]>
git never sets
CURLOPT_SSL_VERIFYSTATUS, so libcurl never requests the OCSP "Certificate Status Request" extension and any stapled response a server sends is ignored, including responses that explicitly state the certificate has been revoked.Quoted 7 lines
Quoted 3 lines
...
This patch was merged to 'next' the other day, and the last test in the new t5585 fails on my system.Sorry about a premature merge. Since we are not in a hurry to take this topic in (or no new feature topic in general), let me revert it out of 'next' and give it a clean slate to try again.
Quoted 7 lines
Well, if you hadn't merged it, we would perhaps still be none the wiser, because, alas, I don't have the bandwidth to run tests on the seen branch regularly...
However, CI does, but I can't seem to find any CI runs that failed because of this, which makes me worried that something is wrong on my end.
Well, I was bisecting between jch..seen, making a wishful assumption that there is only one topic to blame the recent CI timeouts (see [*1*] and [*2*] for examples) that many linux-* jobs spin forever and time out after 6 hours, for the past few days. I finally found out that 'seen' with this topic ejected, even though it seems to fail jobs that depend on older version of Ubuntu, does not exhibit the time-out-after-spinning-for-6-hours symptom ([*3*]).
So tentatively I am ejecting it out of 'seen'. It might be some funny interactions with other topics; I haven't tried to push it alone to see what happens there at CI to test.
[References]
*1* https://github.com/git/git/actions/runs/37377476653
*2* https://github.com/git/git/actions/runs/37568540157
*3* https://github.com/git/git/actions/runs/37662034989
Quoted 11 lines · Junio C Hamano
Well, I was bisecting between jch..seen, making a wishful assumption that there is only one topic to blame the recent CI timeouts (see [*1*] and [*2*] for examples) that many linux-* jobs spin forever and time out after 6 hours, for the past few days. I finally found out that 'seen' with this topic ejected, even though it seems to fail jobs that depend on older version of Ubuntu, does not exhibit the time-out-after-spinning-for-6-hours symptom ([*3*]).
So tentatively I am ejecting it out of 'seen'. It might be some funny interactions with other topics; I haven't tried to push it alone to see what happens there at CI to test.
Now I have tried the topic in isolation, just to double check.
[*4*] is the CI run of the topic alone, queued directly on top of Git 2.56, that timed out many linux-* jobs after 6 hours, which was exactly the breakage I was chasing.
The reason why linux-TEST-vars and linux32 failed is unrelated; they are failing to download long deprecated Ubuntu 20.04 and CI jobs on other branches often fail the same way.