Skip to contentSkip to conversation

pem: refactor to expose pem iterator privately

PATCH Superseded James Prestwood 2 participants 4 messages Oct 1, 2019 – Oct 2, 2019

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/3 pem: refactor to expose pem iterator privately

This refactors pem.c to expose a new internal API, pem_next. This API will allow iteration/parsing of PEM(s). Other APIs in pem.c were updated to use pem_next.

pem_next behaves similar to pem_load_buffer, except it does not do any base64 decoding. Instead it just returns the start of the PEM as a string, and the length as an out parameter. This allows simple PEM iteration loops. Strictly for pem.c usage we decode the base64 string in the loop, but settings.c will need this for parsing PEMs from settings files and returning the PEM as a full string, including the start/end tags.

endp was removed from pem_load_buffer as it was no longer being used.

---
 ell/pem-private.h | 32 ++++++++++++++++++++
 ell/pem.c         | 77 ++++++++++++++++++++++++++++++++---------------
 2 files changed, 85 insertions(+), 24 deletions(-)
 create mode 100644 ell/pem-private.h

-v4:
* Added 'stict' argument to pem_next.

ell/pem-private.h new file +32 -0
new file mode 100644
index 0000000..bb0127f
@@ -0,0 +1,32 @@
+/*
+ *
+ *  Embedded Linux library
+ *
+ *  Copyright (C) 2019  Intel Corporation. All rights reserved.
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Lesser General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2.1 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Lesser General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Lesser General Public
+ *  License along with this library; if not, write to the Free Software
+ *  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ *
+ */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#define _GNU_SOURCE
+#include <sys/types.h>
+
+const char *pem_next(const void *buf, size_t buf_len, char **type_label,
+				size_t *len, size_t *base64_len,
+				const char **endp, bool strict);
ell/pem.c +53 -24

141 lines not on this page · raw message

Signature · 1 line
2.17.1
#1 lore raw

PATCH v4 2/3 settings: introduce extended groups concept

This patch introduces a new extended format for settings files. This new format allows raw data to be provided under a given group rather than key/value pairs. A group using an extended format will contain both the extended type and group name in the format, for example:

[@pem(a)certificate]

Where 'pem' is the type and 'certificate' is the name. The type of format requires an extension specific parser implementation in settings as the parser must know when to start/stop parsing data. This allows for any textual characters to be present after the group is declared, including special characters like [] (its completely up to the parser to decide whats valid and what is not).

Three new l_settings APIs were added:

l_settings_has_embedded_group
l_settings_get_embedded_groups
l_settings_get_embedded_value

In this patch, the 'pem' extension type is also introduced, and for now is the only extended type. The pem extended type expects a base64 encoded PEM (or list of PEMs) after the group. The pem parser will stop after the last PEM is found. Examples can be found in unit/test-settings.c

---
 ell/ell.sym    |   3 +
 ell/settings.c | 234 ++++++++++++++++++++++++++++++++++++++++++++++---
 ell/settings.h |   7 ++
 3 files changed, 233 insertions(+), 11 deletions(-)

-v4:
* Updated to properly detect parsing error vs no more certs (using endp)

ell/ell.sym +3 -0
index e02f0fe..32d94f5 100644
@@ -425,6 +425,9 @@ global:
 	l_settings_set_float;
 	l_settings_remove_group;
 	l_settings_remove_key;
+	l_settings_has_embedded_group;
+	l_settings_get_embedded_groups;
+	l_settings_get_embedded_value;
 	/* signal */
 	l_signal_create;
 	l_signal_remove;
ell/settings.c +223 -11

301 lines not on this page · raw message

ell/settings.h +7 -0
index 0da9f55..e4203af 100644
@@ -123,6 +123,13 @@ bool l_settings_remove_key(struct l_settings *settings, const char *group_name,
 				const char *key);
 bool l_settings_remove_group(struct l_settings *settings,
 				const char *group_name);
+
+char **l_settings_get_embedded_groups(struct l_settings *settings);
+bool l_settings_has_embedded_group(struct l_settings *settings,
+					const char *group);
+const char *l_settings_get_embedded_value(struct l_settings *settings,
+						const char *group_name,
+						const char **out_type);
 #ifdef __cplusplus
 }
 #endif
Signature · 1 line
2.17.1
#2 lore raw

PATCH v4 3/3 unit/test-settings: add pem extended type tests

In reply to James Prestwood
---
 unit/settings.test   |  28 +++++
 unit/test-settings.c | 238 ++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 264 insertions(+), 2 deletions(-)
unit/settings.test +28 -0
index 3884f7d..5f2d4f8 100644
@@ -20,3 +20,31 @@ StringList=Foo,Bar,Baz
 StringListEmpty=
 StringListOne=FooBarBaz
 StringWithSpaces=Bar B Q
+
+[@pem(a)certificate]
+-----BEGIN CERTIFICATE-----
+MIIEajCCA1KgAwIBAgIUKIOXQXEyHZsOFw/+ihDBNGTQnWUwDQYJKoZIhvcNAQEL
+BQAweDE1MDMGA1UECgwsSW50ZXJuYXRpb25hbCBVbmlvbiBvZiBFeGFtcGxlIE9y
+Z2FuaXphdGlvbnMxHzAdBgNVBAMMFkNlcnRpZmljYXRlIGlzc3VlciBndXkxHjAc
+BgkqhkiG9w0BCQEWD2NhQG1haWwuZXhhbXBsZTAeFw0xOTA5MTYxNzEyNThaFw00
+NzAyMDExNzEyNThaMHgxNTAzBgNVBAoMLEludGVybmF0aW9uYWwgVW5pb24gb2Yg
+RXhhbXBsZSBPcmdhbml6YXRpb25zMR8wHQYDVQQDDBZDZXJ0aWZpY2F0ZSBpc3N1
+ZXIgZ3V5MR4wHAYJKoZIhvcNAQkBFg9jYUBtYWlsLmV4YW1wbGUwggEiMA0GCSqG
+SIb3DQEBAQUAA4IBDwAwggEKAoIBAQCjovj3aq26sAQ0k5vD/BVp40p0lhz1+Wet
+1EcdQa1arVIca9nhfvoHfJAmK+zzqQLbvI0/e2if4X6OKf41g7w7VaYS9qv5jZZ0
+v/7aL6PUa2F7C9HG/vuIII/dRvP2uQ43PLxeTeZyj7bBUB9xCFCpzB+7AZuUuH0H
+ABaC9CAGZWImBY5NUXST7E/BsvqU80KJglDovcabthvwoekji9DC/wwISLE1e9cO
+A9IB0Co0mA1ME6wzrawmuTzxUw9BsmEhbKhFGBRwIrrq0r4GvDmeMFiZjXv+I0vq
+wSCyRtgoeBmyemqIEgiN4Z23V7ps3dbYF/tw96Zj7rd5gtjY9VSdAgMBAAGjgesw
+gegwDwYDVR0TBAgwBgEB/wIBADAdBgNVHQ4EFgQUnvllvvxy17VqqR0FfgS4NwzX
+jScwgbUGA1UdIwSBrTCBqoAUnvllvvxy17VqqR0FfgS4NwzXjSehfKR6MHgxNTAz
+BgNVBAoMLEludGVybmF0aW9uYWwgVW5pb24gb2YgRXhhbXBsZSBPcmdhbml6YXRp
+b25zMR8wHQYDVQQDDBZDZXJ0aWZpY2F0ZSBpc3N1ZXIgZ3V5MR4wHAYJKoZIhvcN
+AQkBFg9jYUBtYWlsLmV4YW1wbGWCFCiDl0FxMh2bDhcP/ooQwTRk0J1lMA0GCSqG
+SIb3DQEBCwUAA4IBAQBROAyWfQyKXQ007U6ctgihHbg/lsfEEfeNPG+QRVt8/e53
+4fH6scuY9bW7CZQSdiBo178ITHrIOo2CuFMa0ysnW3V1M9/s0dUYjBHYdpTEEQ+d
+tgm1uRLiTsYeBtueRItEmZU6JjgmvAH8i1UqI0e5iYlfnovPmftpqIwRH7k7A9kS
+SehC9QkkrnIttDEoeYTGhLOJu1Fx2cwAodce6VNgz/k1zIXY5Tprg440zrCwc+th
+MpX48F31ggg8Wd5N6Xg1nricGwL8K90ts6xvwF1WwKsg6BeYdyC0eYBqQ41MA/7P
+DK3OGM6cC5tbQGWaIT0Q407GJBGpaijDicA2YqlK
+-----END CERTIFICATE-----
unit/test-settings.c +236 -2

274 lines not on this page · raw message

Signature · 1 line
2.17.1
#3 lore raw
In reply to James Prestwood

Hi James,

Quoted 7 lines · James Prestwood

This refactors pem.c to expose a new internal API, pem_next. This API will allow iteration/parsing of PEM(s). Other APIs in pem.c were updated to use pem_next.

pem_next behaves similar to pem_load_buffer, except it does not do any base64 decoding. Instead it just returns the start of the PEM as a string, and the length as an out parameter. This allows simple

But it returns the start of the base64 data?

Quoted 42 lines

PEM iteration loops. Strictly for pem.c usage we decode the base64 string in the loop, but settings.c will need this for parsing PEMs from settings files and returning the PEM as a full string, including the start/end tags.

endp was removed from pem_load_buffer as it was no longer being used.

---
  ell/pem-private.h | 32 ++++++++++++++++++++
  ell/pem.c         | 77 ++++++++++++++++++++++++++++++++---------------
  2 files changed, 85 insertions(+), 24 deletions(-)

create mode 100644 ell/pem-private.h

(quoted diff) +0 -1
-v4:
  * Added 'stict' argument to pem_next.
ell/pem-private.h new file +21 -0
new file mode 100644
index 0000000..bb0127f
@@ -0,0 +1,32 @@
+/*
+ *
+ *  Embedded Linux library
+ *
+ *  Copyright (C) 2019  Intel Corporation. All rights reserved.
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Lesser General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2.1 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Lesser General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Lesser General Public
+ *  License along with this library; if not, write to the Free Software
+ *  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ *
+ */

ell/pem-private.h line 31

Quoted 10 lines
ell/pem-private.h new file +10 -0
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#define _GNU_SOURCE
+#include <sys/types.h>
+
+const char *pem_next(const void *buf, size_t buf_len, char **type_label,
+				size_t *len, size_t *base64_len,

Why do you need the len parameter when you have endp?

Quoted 56 lines
ell/pem-private.h new file +1 -0
+				const char **endp, bool strict);
ell/pem.c +12 -14
index c2bf7ba..79e4d07 100644
@@ -45,6 +45,7 @@
  #include "cipher.h"
  #include "cert-private.h"
  #include "missing.h"
+#include "pem-private.h"
 
  #define PEM_START_BOUNDARY	"-----BEGIN "
  #define PEM_END_BOUNDARY	"-----END "
@@ -126,15 +127,15 @@ static bool is_end_boundary(const void *buf, size_t buf_len,
  	return true;
  }
 
-static uint8_t *pem_load_buffer(const void *buf, size_t buf_len,
-				char **type_label, size_t *len,
-				const uint8_t **endp)
+const char *pem_next(const void *buf, size_t buf_len, char **type_label,
+				size_t *len, size_t *base64_len,
+				const char **endp, bool strict)
  {
  	const char *buf_ptr = buf;
  	const char *base64_data = NULL, *eol;
  	const char *label = NULL;
-	uint8_t *data;
  	size_t label_len = 0;
+	const char *start = NULL;
 
  	/*
  	 * The base64 parser uses the RFC7468 laxbase64text grammar but we
@@ -148,26 +149,28 @@ static uint8_t *pem_load_buffer(const void *buf, size_t buf_len,
  				break;
 
  		if (!base64_data) {
-			label = is_start_boundary(buf, eol - buf_ptr,
+			label = is_start_boundary(buf_ptr, eol - buf_ptr,
  							&label_len);
-
-			if (label)
+			if (label) {
+				start = label - strlen("-----BEGIN ");
  				base64_data = eol;
-		} else if (is_end_boundary(buf_ptr, eol - buf_ptr, label,
-							label_len)) {
-			data = l_base64_decode(
-					(const char *) base64_data,
-					buf_ptr - base64_data, len);
-			if (!data)
-				return NULL;
+			} else if (strict)
+				break;
+		} else if (start && is_end_boundary(buf_ptr, eol - buf_ptr,
+							label, label_len)) {

ell/pem.c static uint8_t *pem_load_buffer(const void *buf, size_t buf_len, · line 168

Quoted 10 lines
ell/pem.c +7 -2
+			if (len)
+				*len = eol - start + 1;
 
-			*type_label = l_strndup((const char *) label,
-							label_len);
+			if (type_label)
+				*type_label = l_strndup(label, label_len);
+
+			if (base64_len)
+				*base64_len = buf_ptr - base64_data - 1;

Why -1?

Quoted 58 lines
ell/pem.c +19 -5
  			if (endp)
-				*endp = (uint8_t *)eol + 1;
+				*endp = eol + 1;
 
-			return data;
+			return base64_data;
  		}
 
  		if (eol == buf_ptr + buf_len)
@@ -189,10 +192,24 @@ static uint8_t *pem_load_buffer(const void *buf, size_t buf_len,
  	return NULL;
  }
 
+static uint8_t *pem_load_buffer(const void *buf, size_t buf_len,
+				char **type_label, size_t *len)
+{
+	size_t base64_len;
+	const char *base64;
+
+	base64 = pem_next(buf, buf_len, type_label, NULL, &base64_len,
+				NULL, false);
+	if (!base64)
+		return NULL;
+
+	return l_base64_decode(base64, base64_len, len);
+}
+
  LIB_EXPORT uint8_t *l_pem_load_buffer(const void *buf, size_t buf_len,
  					char **type_label, size_t *out_len)
  {
-	return pem_load_buffer(buf, buf_len, type_label, out_len, NULL);
+	return pem_load_buffer(buf, buf_len, type_label, out_len);
  }
 
  struct pem_file_info {
@@ -242,7 +259,7 @@ LIB_EXPORT uint8_t *l_pem_load_file(const char *filename,
  		return NULL;
 
  	result = pem_load_buffer(file.data, file.st.st_size,
-					type_label, len, NULL);
+					type_label, len);
  	pem_file_close(&file);
  	return result;
  }
@@ -288,7 +305,7 @@ LIB_EXPORT struct l_certchain *l_pem_load_certificate_chain(
  LIB_EXPORT struct l_queue *l_pem_load_certificate_list_from_data(
  						const void *buf, size_t len)
  {
-	const uint8_t *ptr, *end;
+	const char *ptr, *end;
  	struct l_queue *list = NULL;
 
  	ptr = buf;
@@ -297,10 +314,22 @@ LIB_EXPORT struct l_queue *l_pem_load_certificate_list_from_data(
  	while (ptr && ptr < end) {
  		uint8_t *der;
  		size_t der_len;

ell/pem.c LIB_EXPORT struct l_queue *l_pem_load_certificate_list_from_data( · line 325

Quoted 10 lines
ell/pem.c +8 -1
-		char *label;
+		char *label = NULL;
  		struct l_cert *cert;
+		size_t out_len = 0;
+		const char *base64;
+		size_t base64_len;
+
+		base64 = pem_next(ptr, len, &label, &out_len, &base64_len,
+					&ptr, false);
+		/* if ptr is set but we got NULL, this was a parse error */

The comment is a little misleading. It should be if ptr was not reset to NULL.

e.g.

if (!base64) {
	if (!ptr) // EOF
		break;

	goto error;
}

ell/pem.c LIB_EXPORT struct l_queue *l_pem_load_certificate_list_from_data( · line 335

Quoted 11 lines
ell/pem.c +6 -1
+		if (!base64 && ptr)
+			goto error;
+
+		if (!base64)
+			break;
 
-		der = pem_load_buffer(ptr, end - ptr, &label, &der_len, &ptr);
+		der = l_base64_decode(base64, base64_len, &der_len);
 
  		if (!der || strcmp(label, "CERTIFICATE")) {
  			if (der)

This part still looks wrong since you're not taking out the existing EOF handling code below...

ell/pem.c LIB_EXPORT struct l_key *l_pem_load_private_key_from_data(const void *buf, · line 505

Quoted 10 lines
ell/pem.c +1 -1
@@ -470,7 +499,7 @@ LIB_EXPORT struct l_key *l_pem_load_private_key_from_data(const void *buf,
  	if (encrypted)
  		*encrypted = false;
 
-	content = pem_load_buffer(buf, buf_len, &label, &len, NULL);
+	content = pem_load_buffer(buf, buf_len, &label, &len);
 
  	if (!content)
  		return NULL;

Regards,
-Denis