Skip to contentSkip to conversation

Extended group format

PATCH In review James Prestwood 2 participants 8 messages Sep 27, 2019

lore
Share & tools
lore
#0 lore raw

cover Extended group format

Most of the details are explained in the patch notes, but the main reasoning for this is to allow embedding PEMs inside an l_settings file. We have found that some systems are provisioned with certificates in /home/<user>/.cert/. System services are not really intended to have access to /home (though this is not always the case), and for IWD we do not want to allow that. For this reason it makes a lot of sense to include the certs in the provisioning file. This also makes provisioning much easier from both an admin and user perspective as you just need to supply a single file rather than the network config plus certificates/keys.

James Prestwood (5):
pem: add _from_data variants to pem APIs unit/test-pem: add tests for new _from_data APIs
pem: expose is_{start,end}_boundary
settings: introduce extended groups concept unit/test-settings: add pem extended type tests

 ell/ell.sym          |   5 +
 ell/pem.c            | 152 ++++++++++++++++++++--------
 ell/pem.h            |   7 ++
 ell/settings.c       | 231 +++++++++++++++++++++++++++++++++++++++++-
 ell/settings.h       |   5 +
 pem-private.h        |  33 ++++++
 unit/settings.test   |  28 ++++++
 unit/test-pem.c      | 163 ++++++++++++++++++++++++++++++
 unit/test-settings.c | 234 ++++++++++++++++++++++++++++++++++++++++++-
 9 files changed, 806 insertions(+), 52 deletions(-)
 create mode 100644 pem-private.h
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/5 pem: add _from_data variants to pem APIs

Loading cert chains/lists, and private keys was only possible to do from a file, or as a raw buffer (e.g. from the network). This patch adds 3 new APIs:

l_pem_load_certificate_chain_from_data
l_pem_load_certificate_list_from_data
l_pem_load_private_key_from_data

These expect the input data just as if it was read in from a PEM file (BEGIN/END included).

---
 ell/ell.sym |   3 ++
 ell/pem.c   | 141 +++++++++++++++++++++++++++++++++++++---------------
 ell/pem.h   |   7 +++
 3 files changed, 111 insertions(+), 40 deletions(-)
ell/ell.sym +3 -0
index 709a0b4..e02f0fe 100644
@@ -362,9 +362,12 @@ global:
 	/* pem */
 	l_pem_load_buffer;
 	l_pem_load_certificate_chain;
+	l_pem_load_certificate_chain_from_data;
 	l_pem_load_certificate_list;
+	l_pem_load_certificate_list_from_data;
 	l_pem_load_file;
 	l_pem_load_private_key;
+	l_pem_load_private_key_from_data;
 	/* pkcs5 */
 	l_pkcs5_pbkdf1;
 	l_pkcs5_pbkdf2;
ell/pem.c +101 -40

178 lines not on this page · raw message

ell/pem.h +7 -0
index fc5b019..41583e2 100644
@@ -38,11 +38,18 @@ uint8_t *l_pem_load_file(const char *filename, int index,
 				char **type_label, size_t *len);
 
 struct l_certchain *l_pem_load_certificate_chain(const char *filename);
+struct l_certchain *l_pem_load_certificate_chain_from_data(const void *buf,
+								size_t len);
 struct l_queue *l_pem_load_certificate_list(const char *filename);
+struct l_queue *l_pem_load_certificate_list_from_data(const void *buf,
+							size_t len);
 
 struct l_key *l_pem_load_private_key(const char *filename,
 					const char *passphrase,
 					bool *encrypted);
+struct l_key *l_pem_load_private_key_from_data(const void *buf, size_t len,
+						const char *passphrase,
+						bool *encrypted);
 
 #ifdef __cplusplus
 }
Signature · 1 line
2.17.1
#2 lore raw

Re: [PATCH 1/5] pem: add _from_data variants to pem APIs

Hi James,

Quoted 8 lines · James Prestwood

Loading cert chains/lists, and private keys was only possible to do from a file, or as a raw buffer (e.g. from the network). This patch adds 3 new APIs:

l_pem_load_certificate_chain_from_data
l_pem_load_certificate_list_from_data
l_pem_load_private_key_from_data

Quoted 8 lines

These expect the input data just as if it was read in from a PEM file (BEGIN/END included).

---
  ell/ell.sym |   3 ++
  ell/pem.c   | 141 +++++++++++++++++++++++++++++++++++++---------------
  ell/pem.h   |   7 +++
  3 files changed, 111 insertions(+), 40 deletions(-)

Patches 1 & 2 applied, thanks.

Regards,
-Denis

#3 lore raw

PATCH 2/5 unit/test-pem: add tests for new _from_data APIs

In reply to James Prestwood

Existing certs/keys were stripped of new lines and stored as strings in the test. These new tests just verify that the certs load/verify and that a private key can be decrypted.

---
 unit/test-pem.c | 163 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 163 insertions(+)
unit/test-pem.c +163 -0
index 085e061..df396ce 100644
@@ -62,6 +62,166 @@ static const struct pem_test empty_label = {
 	.decoded_size = 6,
 };
 
+struct pem_from_data_test {
+	const char *list;
+	const char *ca;
+};
+
+static const struct pem_from_data_test single_line_cert_chain = {
+	/* Copied from cert-chain.pem */
+	.list =
+		"-----BEGIN CERTIFICATE-----\n"
+		"MIIEXDCCA0SgAwIBAgIJALjNE85c9plgMA0GCSqGSIb3DQEBCwUAMHgxNTAzBgNV\n"
+		"BAoMLEludGVybmF0aW9uYWwgVW5pb24gb2YgRXhhbXBsZSBPcmdhbml6YXRpb25z\n"
+		"MR8wHQYDVQQDDBZDZXJ0aWZpY2F0ZSBpc3N1ZXIgZ3V5MR4wHAYJKoZIhvcNAQkB\n"
+		"Fg9jYUBtYWlsLmV4YW1wbGUwHhcNMTkwOTE2MTcxMzAzWhcNNDcwMjAxMTcxMzAz\n"
+		"WjB4MTUwMwYDVQQKDCxJbnRlcm5hdGlvbmFsIFVuaW9uIG9mIEV4YW1wbGUgT3Jn\n"
+		"YW5pemF0aW9uczEfMB0GA1UEAwwWQ2VydGlmaWNhdGUgaXNzdWVyIGd1eTEeMBwG\n"
+		"CSqGSIb3DQEJARYPY2FAbWFpbC5leGFtcGxlMIIBIjANBgkqhkiG9w0BAQEFAAOC\n"
+		"AQ8AMIIBCgKCAQEA7Lft5O6BtUUokuueQ7mBQVzRzPeH0Nl3NjgGnfBYcz7O2Jca\n"
+		"rFSBPsV76reUG4QFQudsdwyaLOpniFFSFaI3GRXMxjwZJJjLqvT0aebTiLUSKseA\n"
+		"QkP/NSITmIljs2yclnPJGIApLuFvykPagx+yc9ckbziEz1PvKB/ukbiU/zt6QCru\n"
+		"BbyCQ1kWBuyrS3RC0/UgmrSbL7YkkmuD2B1vyZLIoPsJijXs2GJQY3a+zpLemTth\n"
+		"i/Vw4AURJS1gfEUDNzf9Y9+o7vWJfzk+g7xm1XpMTsNTd7q6UwHOi1xdiKCEPT+q\n"
+		"c3LXi7qgWqSXeD+F513PM3JMJ3Wk1H8K4VwJwQIDAQABo4HoMIHlMAwGA1UdEwQF\n"
+		"MAMBAf8wHQYDVR0OBBYEFMuhnjqw8YGMg0cyYlQppMncWis/MIG1BgNVHSMEga0w\n"
+		"gaqAFJ75Zb78cte1aqkdBX4EuDcM140noXykejB4MTUwMwYDVQQKDCxJbnRlcm5h\n"
+		"dGlvbmFsIFVuaW9uIG9mIEV4YW1wbGUgT3JnYW5pemF0aW9uczEfMB0GA1UEAwwW\n"
+		"Q2VydGlmaWNhdGUgaXNzdWVyIGd1eTEeMBwGCSqGSIb3DQEJARYPY2FAbWFpbC5l\n"
+		"eGFtcGxlghQog5dBcTIdmw4XD/6KEME0ZNCdZTANBgkqhkiG9w0BAQsFAAOCAQEA\n"
+		"PjX5n/fgkskZmh9aRhX8r9985JtxMdgogJP4uwRbfuQPzAqYyu9QlAOcRl6tNGN7\n"
+		"mztB5RfJ9HDyjS9iGXsvKXS8wT5ELbuATev+C1Ppxakd3gvJMN4ZqYn32JqRYigN\n"
+		"L2V2jo9RzVUuFa3YP6sw0KfZAfHsfUmQCxAm8HAfQg98aYyIXu/OzeVUsAuhfqWN\n"
+		"qvWcOLjTQTn6t10OHHdIYw59EpIEOPD3Opq7pLgIm+EV3eVMWthSLYbEhIavh8Pc\n"
+		"xN9lqCg887kTawbXbXd49Z8jYZxjxQl7IoonvIyrPhhabKjKCpE2bRFzzpia0PkC\n"
+		"fRgh+KB2tqIeAoekDllmbA==\n"
+		"-----END CERTIFICATE-----\n"
+		"-----BEGIN CERTIFICATE-----\n"
+		"MIIEajCCA1KgAwIBAgIUKIOXQXEyHZsOFw/+ihDBNGTQnWUwDQYJKoZIhvcNAQEL\n"
+		"BQAweDE1MDMGA1UECgwsSW50ZXJuYXRpb25hbCBVbmlvbiBvZiBFeGFtcGxlIE9y\n"
+		"Z2FuaXphdGlvbnMxHzAdBgNVBAMMFkNlcnRpZmljYXRlIGlzc3VlciBndXkxHjAc\n"
+		"BgkqhkiG9w0BCQEWD2NhQG1haWwuZXhhbXBsZTAeFw0xOTA5MTYxNzEyNThaFw00\n"
+		"NzAyMDExNzEyNThaMHgxNTAzBgNVBAoMLEludGVybmF0aW9uYWwgVW5pb24gb2Yg\n"
+		"RXhhbXBsZSBPcmdhbml6YXRpb25zMR8wHQYDVQQDDBZDZXJ0aWZpY2F0ZSBpc3N1\n"
+		"ZXIgZ3V5MR4wHAYJKoZIhvcNAQkBFg9jYUBtYWlsLmV4YW1wbGUwggEiMA0GCSqG\n"
+		"SIb3DQEBAQUAA4IBDwAwggEKAoIBAQCjovj3aq26sAQ0k5vD/BVp40p0lhz1+Wet\n"
+		"1EcdQa1arVIca9nhfvoHfJAmK+zzqQLbvI0/e2if4X6OKf41g7w7VaYS9qv5jZZ0\n"
+		"v/7aL6PUa2F7C9HG/vuIII/dRvP2uQ43PLxeTeZyj7bBUB9xCFCpzB+7AZuUuH0H\n"
+		"ABaC9CAGZWImBY5NUXST7E/BsvqU80KJglDovcabthvwoekji9DC/wwISLE1e9cO\n"
+		"A9IB0Co0mA1ME6wzrawmuTzxUw9BsmEhbKhFGBRwIrrq0r4GvDmeMFiZjXv+I0vq\n"
+		"wSCyRtgoeBmyemqIEgiN4Z23V7ps3dbYF/tw96Zj7rd5gtjY9VSdAgMBAAGjgesw\n"
+		"gegwDwYDVR0TBAgwBgEB/wIBADAdBgNVHQ4EFgQUnvllvvxy17VqqR0FfgS4NwzX\n"
+		"jScwgbUGA1UdIwSBrTCBqoAUnvllvvxy17VqqR0FfgS4NwzXjSehfKR6MHgxNTAz\n"
+		"BgNVBAoMLEludGVybmF0aW9uYWwgVW5pb24gb2YgRXhhbXBsZSBPcmdhbml6YXRp\n"
+		"b25zMR8wHQYDVQQDDBZDZXJ0aWZpY2F0ZSBpc3N1ZXIgZ3V5MR4wHAYJKoZIhvcN\n"
+		"AQkBFg9jYUBtYWlsLmV4YW1wbGWCFCiDl0FxMh2bDhcP/ooQwTRk0J1lMA0GCSqG\n"
+		"SIb3DQEBCwUAA4IBAQBROAyWfQyKXQ007U6ctgihHbg/lsfEEfeNPG+QRVt8/e53\n"
+		"4fH6scuY9bW7CZQSdiBo178ITHrIOo2CuFMa0ysnW3V1M9/s0dUYjBHYdpTEEQ+d\n"
+		"tgm1uRLiTsYeBtueRItEmZU6JjgmvAH8i1UqI0e5iYlfnovPmftpqIwRH7k7A9kS\n"
+		"SehC9QkkrnIttDEoeYTGhLOJu1Fx2cwAodce6VNgz/k1zIXY5Tprg440zrCwc+th\n"
+		"MpX48F31ggg8Wd5N6Xg1nricGwL8K90ts6xvwF1WwKsg6BeYdyC0eYBqQ41MA/7P\n"
+		"DK3OGM6cC5tbQGWaIT0Q407GJBGpaijDicA2YqlK\n"
+		"-----END CERTIFICATE-----\n",
+	/* copied from cert-ca.pem */
+	.ca =	"-----BEGIN CERTIFICATE-----\n"
+		"MIIEajCCA1KgAwIBAgIUKIOXQXEyHZsOFw/+ihDBNGTQnWUwDQYJKoZIhvcNAQEL\n"
+		"BQAweDE1MDMGA1UECgwsSW50ZXJuYXRpb25hbCBVbmlvbiBvZiBFeGFtcGxlIE9y\n"
+		"Z2FuaXphdGlvbnMxHzAdBgNVBAMMFkNlcnRpZmljYXRlIGlzc3VlciBndXkxHjAc\n"
+		"BgkqhkiG9w0BCQEWD2NhQG1haWwuZXhhbXBsZTAeFw0xOTA5MTYxNzEyNThaFw00\n"
+		"NzAyMDExNzEyNThaMHgxNTAzBgNVBAoMLEludGVybmF0aW9uYWwgVW5pb24gb2Yg\n"
+		"RXhhbXBsZSBPcmdhbml6YXRpb25zMR8wHQYDVQQDDBZDZXJ0aWZpY2F0ZSBpc3N1\n"
+		"ZXIgZ3V5MR4wHAYJKoZIhvcNAQkBFg9jYUBtYWlsLmV4YW1wbGUwggEiMA0GCSqG\n"
+		"SIb3DQEBAQUAA4IBDwAwggEKAoIBAQCjovj3aq26sAQ0k5vD/BVp40p0lhz1+Wet\n"
+		"1EcdQa1arVIca9nhfvoHfJAmK+zzqQLbvI0/e2if4X6OKf41g7w7VaYS9qv5jZZ0\n"
+		"v/7aL6PUa2F7C9HG/vuIII/dRvP2uQ43PLxeTeZyj7bBUB9xCFCpzB+7AZuUuH0H\n"
+		"ABaC9CAGZWImBY5NUXST7E/BsvqU80KJglDovcabthvwoekji9DC/wwISLE1e9cO\n"
+		"A9IB0Co0mA1ME6wzrawmuTzxUw9BsmEhbKhFGBRwIrrq0r4GvDmeMFiZjXv+I0vq\n"
+		"wSCyRtgoeBmyemqIEgiN4Z23V7ps3dbYF/tw96Zj7rd5gtjY9VSdAgMBAAGjgesw\n"
+		"gegwDwYDVR0TBAgwBgEB/wIBADAdBgNVHQ4EFgQUnvllvvxy17VqqR0FfgS4NwzX\n"
+		"jScwgbUGA1UdIwSBrTCBqoAUnvllvvxy17VqqR0FfgS4NwzXjSehfKR6MHgxNTAz\n"
+		"BgNVBAoMLEludGVybmF0aW9uYWwgVW5pb24gb2YgRXhhbXBsZSBPcmdhbml6YXRp\n"
+		"b25zMR8wHQYDVQQDDBZDZXJ0aWZpY2F0ZSBpc3N1ZXIgZ3V5MR4wHAYJKoZIhvcN\n"
+		"AQkBFg9jYUBtYWlsLmV4YW1wbGWCFCiDl0FxMh2bDhcP/ooQwTRk0J1lMA0GCSqG\n"
+		"SIb3DQEBCwUAA4IBAQBROAyWfQyKXQ007U6ctgihHbg/lsfEEfeNPG+QRVt8/e53\n"
+		"4fH6scuY9bW7CZQSdiBo178ITHrIOo2CuFMa0ysnW3V1M9/s0dUYjBHYdpTEEQ+d\n"
+		"tgm1uRLiTsYeBtueRItEmZU6JjgmvAH8i1UqI0e5iYlfnovPmftpqIwRH7k7A9kS\n"
+		"SehC9QkkrnIttDEoeYTGhLOJu1Fx2cwAodce6VNgz/k1zIXY5Tprg440zrCwc+th\n"
+		"MpX48F31ggg8Wd5N6Xg1nricGwL8K90ts6xvwF1WwKsg6BeYdyC0eYBqQ41MA/7P\n"
+		"DK3OGM6cC5tbQGWaIT0Q407GJBGpaijDicA2YqlK\n"
+		"-----END CERTIFICATE-----\n",
+};
+
+static void destroy_cert(void *cert)
+{
+	l_cert_free(cert);
+}
+
+static void test_chain_from_data(const void *data)
+{
+	const struct pem_from_data_test *test = data;
+	struct l_queue *twocas;
+	struct l_certchain *chain;
+
+	twocas = l_pem_load_certificate_list_from_data(test->list,
+							strlen(test->list));
+	assert(twocas);
+
+	chain = l_pem_load_certificate_chain_from_data(test->ca,
+							strlen(test->ca));
+	assert(chain);
+
+	assert(l_certchain_verify(chain, twocas, NULL));
+
+	l_certchain_free(chain);
+	l_queue_destroy(twocas, destroy_cert);
+}
+
+static void test_priv_key_from_data(const void *data)
+{
+	bool is_encrypted = false;
+	struct l_key *key;
+
+	const char *raw_private_key =
+		"-----BEGIN ENCRYPTED PRIVATE KEY-----\n"
+		"MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkqhkiG9w0BBQwwHAQIvjkVXsNnUUgCAggA\n"
+		"MAwGCCqGSIb3DQILBQAwHQYJYIZIAWUDBAEqBBAQfXcH4tJZzrKM0bmpXyQWBIIE\n"
+		"0FwZdv9kfXAZVbPIC2UZLpAqrFqxaaxPMA7FxZrS2sI7QmkXEIfO5TkR8IupYigh\n"
+		"s/41jv7V5Mij1syrSodfiYDq3Y0gb9tF9Cb0FNoJwJ9f29X/h1GgnG5NPQBQEH4d\n"
+		"zkqCA8Q8tzh8UTGXLcPwKYSmsAK9Rq739qre5qwHY0+hcoCtUfrev4twFUSC/PUj\n"
+		"oJDFUxQyVt+WCjcuOG+ugWZSENJJe2O8pAqmt7ChuNKGZTe0UEFn/pxgAAgQYfaz\n"
+		"lH/Nx7OQBSVqxdVkFr03/j8eeBy/SzZubirThd0aehwsQTw5/M9rSX8p2ldyjUWF\n"
+		"Fb+UjXFFWdOs21rZtO0LcbdZlIVK94mswI4zo+Vv3f7DsAZPgW+Y36UJbzZNtxRl\n"
+		"C8t97KH3NozGZIq0znC3CmdYk3EsIlMasp1vgyIpjnsyZcCVtCqbl2+PORv4gZyA\n"
+		"9/PMNDNGambIERa4WCLc+Sx5lTryK6wNzQXCMigrpB7yaD+s2CA4OxvdU99iMQzD\n"
+		"9/7cRvEQn/qFhcdTpz3wt97Gs51A+IleJbj9l/50sEsfQmcLVlUM3VbKtozUkaV1\n"
+		"+5/O15HtMQp0jsjwTlz1AzW5eanPIGoFzLiHKfauzrO5L3i5I2G9GGeCtbUV0+Ts\n"
+		"CTwT2kCUnypaNl4D5qdtxe3h+78uW3Yz0f5t4Yw/RlYVSJQZ7irdi3QTgDEEBrpL\n"
+		"pOXTd8nRNxZ+zJZ5ifnBB0Ed+cMxmyKcliVnVLSV0KseNn3tKZwmRUtMBiPqKUD1\n"
+		"qh8KskfJ0ye8jdcWIubP/gvDh5OgkSz1OdDZKH/RmkktUWCJoyXOMxIz+7GH9u3n\n"
+		"n9Z6uAteNTefTJyawA3dwlGvRhySAI2nMl2Aj0g+6/ztpUUjXVx09oxZqh9Bn9k4\n"
+		"t+gKaf4osH51QcKFs8J2YcYCwEYilzRAUwyw65Bo/k4myNXA5t2xSWfQIYRY+Yob\n"
+		"pmbhOfDMLY1spEVHQ49hXvKE99eP5dyA0CmwZw2gkbXCYBEE1IPthJGYxO4zZdrq\n"
+		"AYZq22L+09o0899pnD+p/eDTwKaFenjHVqO71khXurF6q7EPz9m4SkphDSNe/9Tc\n"
+		"O11yMrQE9OUBTTd3zYuN8KuZpj2aW2p5/Z7pqCYJTDwlV/+HRmS/8aJ/sgHfYXpS\n"
+		"Wpl/SHav6qI7fE5BlKwOwWE6O+vf0Nm9AMsbMErXTFdXe5dAin/uNuFyJM3bTHVO\n"
+		"SR/R7/zsNoMJwsgogGMSiFbG1ebcSTgMNHKMFS8RvCBNX44fErW2r0bfNjHU4GgO\n"
+		"KJFukksz/6tNfpIi9lU0Xojc7W8CJVdA9RTx8+LClM5nwFQlqyfIrtEXUK5BM+Vz\n"
+		"2OI8DlMTpp0+JbSAdE3z1i8cEDFmbfaJ2pNX/1M0JPfcZmZsJiMtNC5Fn6MFBQME\n"
+		"Fu1MyJuUr+maOqPLb6c4aYa7gVWpiRwwK8nTe1FofKeEY7mi7PyNJI7pARIDmoD4\n"
+		"d5yFZ9Itg/5/XK7GfuRdve1m5/YGpV+u3HWqDnk/xBJ5FhyF9aIPzROYhXkRkVZz\n"
+		"rn7DSN3XL2XXtUMle9++kRNmjB8h9GGn4ljunjs9YJBVTb1Y9C9vH1xLh2hknL4M\n"
+		"h+XY4w5Os5FZNEkIQd/0gLUwgQRK5+j3aetp085GutPR\n"
+		"-----END ENCRYPTED PRIVATE KEY-----\n";
+
+	key = l_pem_load_private_key_from_data(raw_private_key,
+						strlen(raw_private_key),
+						"abc", &is_encrypted);
+	assert(key);
+	assert(is_encrypted);
+
+	l_key_free(key);
+}
+
 static void test_pem(const void *data)
 {
 	const struct pem_test *test = data;
@@ -144,6 +304,9 @@ int main(int argc, char *argv[])
 	l_test_add("pem/empty", test_pem, &empty);
 
 	l_test_add("pem/empty label", test_pem, &empty_label);
+	l_test_add("pem/cert chain from data", test_chain_from_data,
+			&single_line_cert_chain);
+	l_test_add("pem/private key from data", test_priv_key_from_data, NULL);
 
 	if (!l_checksum_is_supported(L_CHECKSUM_MD5, false) ||
 			!l_checksum_is_supported(L_CHECKSUM_SHA1, false) ||
Signature · 1 line
2.17.1
#4 lore raw

PATCH 3/5 pem: expose is_{start,end}_boundary

In reply to James Prestwood

Added a new file pem-private.h which exposes these two APIs as pem_is_start_boundary and pem_is_end_boundary. These will be used in settings.c for extended group PEM parsing.

---
 ell/pem.c     | 11 +++++++----
 pem-private.h | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 4 deletions(-)
 create mode 100644 pem-private.h
ell/pem.c +7 -4
index e268f50..7040cd4 100644
@@ -45,11 +45,12 @@
 #include "cipher.h"
 #include "cert-private.h"
 #include "missing.h"
+#include "pem-private.h"
 
 #define PEM_START_BOUNDARY	"-----BEGIN "
 #define PEM_END_BOUNDARY	"-----END "
 
-static const uint8_t *is_start_boundary(const uint8_t *buf, size_t buf_len,
+const uint8_t *pem_is_start_boundary(const uint8_t *buf, size_t buf_len,
 					size_t *label_len)
 {
 	const uint8_t *start, *end, *ptr;
@@ -102,7 +103,7 @@ static const uint8_t *is_start_boundary(const uint8_t *buf, size_t buf_len,
 	return start;
 }
 
-static bool is_end_boundary(const uint8_t *buf, size_t buf_len,
+bool pem_is_end_boundary(const uint8_t *buf, size_t buf_len,
 				const uint8_t *label, size_t label_len)
 {
 	size_t len = strlen(PEM_END_BOUNDARY) + label_len + 5;
@@ -144,11 +145,13 @@ static uint8_t *pem_load_buffer(const uint8_t *buf, size_t buf_len, int index,
 				break;
 
 		if (!base64_data) {
-			label = is_start_boundary(buf, eol - buf, &label_len);
+			label = pem_is_start_boundary(buf, eol - buf,
+							&label_len);
 
 			if (label)
 				base64_data = eol;
-		} else if (is_end_boundary(buf, eol - buf, label, label_len)) {
+		} else if (pem_is_end_boundary(buf, eol - buf, label,
+							label_len)) {
 			if (index == 0) {
 				data = l_base64_decode(
 						(const char *) base64_data,
pem-private.h new file +33 -0
new file mode 100644
index 0000000..9203881
@@ -0,0 +1,33 @@
+/*
+ *
+ *  Embedded Linux library
+ *
+ *  Copyright (C) 2019  Intel Corporation. All rights reserved.
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Lesser General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2.1 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Lesser General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Lesser General Public
+ *  License along with this library; if not, write to the Free Software
+ *  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ *
+ */
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif
+
+#define _GNU_SOURCE
+#include <sys/types.h>
+
+const uint8_t *pem_is_start_boundary(const uint8_t *buf, size_t buf_len,
+					size_t *label_len);
+bool pem_is_end_boundary(const uint8_t *buf, size_t buf_len,
+				const uint8_t *label, size_t label_len);
Signature · 1 line
2.17.1
#5 lore raw

PATCH 4/5 settings: introduce extended groups concept

In reply to James Prestwood

This patch introduces a new extended format for settings files. This new format allows raw data to be provided under a given group rather than key/value pairs. A group using an extended format will contain both the extended type and group name in the format, for example:

[@pem(a)certificate]

Where 'pem' is the type and 'certificate' is the name. The type of format requries an extension specific parser implementation in settings as the parser must know when to start/stop parsing data. This allows for any textual characters to be present after the group is declared, including special characters like [] (its completely up to the parser to decide whats valid and what is not).

Two new l_settings APIs were added:

l_settings_is_extended_group
l_settings_get_extended_data

In this patch, the 'pem' extension type is also introduced, and for now is the only extended type. The pem extended type expects a base64 encoded PEM (or list of PEMs) after the group. The pem parser will stop after the last PEM is found. Examples can be found in unit/test-settings.c

---
 ell/ell.sym    |   2 +
 ell/settings.c | 231 +++++++++++++++++++++++++++++++++++++++++++++++--
 ell/settings.h |   5 ++
 3 files changed, 233 insertions(+), 5 deletions(-)
ell/ell.sym +2 -0
index e02f0fe..13e3fd8 100644
@@ -425,6 +425,8 @@ global:
 	l_settings_set_float;
 	l_settings_remove_group;
 	l_settings_remove_key;
+	l_settings_is_extended_group;
+	l_settings_get_extended_data;
 	/* signal */
 	l_signal_create;
 	l_signal_remove;
ell/settings.c +226 -5

283 lines not on this page · raw message

ell/settings.h +5 -0
index 0da9f55..bf7bc44 100644
@@ -123,6 +123,11 @@ bool l_settings_remove_key(struct l_settings *settings, const char *group_name,
 				const char *key);
 bool l_settings_remove_group(struct l_settings *settings,
 				const char *group_name);
+
+bool l_settings_is_extended_group(struct l_settings *settings,
+					const char *group);
+const char *l_settings_get_extended_data(struct l_settings *settings,
+						const char *group);
 #ifdef __cplusplus
 }
 #endif
Signature · 1 line
2.17.1
#6 lore raw

Re: [PATCH 4/5] settings: introduce extended groups concept

Hi James,

Quoted 9 lines · James Prestwood

This patch introduces a new extended format for settings files. This new format allows raw data to be provided under a given group rather than key/value pairs. A group using an extended format will contain both the extended type and group name in the format, for example:

[@pem(a)certificate]

Where 'pem' is the type and 'certificate' is the name. The type of format requries an extension specific parser implementation in settings

typo: requires

Quoted 20 lines

as the parser must know when to start/stop parsing data. This allows for any textual characters to be present after the group is declared, including special characters like [] (its completely up to the parser to decide whats valid and what is not).

Two new l_settings APIs were added:

l_settings_is_extended_group
l_settings_get_extended_data

In this patch, the 'pem' extension type is also introduced, and for now is the only extended type. The pem extended type expects a base64 encoded PEM (or list of PEMs) after the group. The pem parser will stop after the last PEM is found. Examples can be found in unit/test-settings.c

---
  ell/ell.sym    |   2 +
  ell/settings.c | 231 +++++++++++++++++++++++++++++++++++++++++++++++--
  ell/settings.h |   5 ++
  3 files changed, 233 insertions(+), 5 deletions(-)

<snip>

Quoted 13 lines
ell/settings.c +1 -0
index 98f0cfd..b8f61b4 100644
@@ -48,6 +48,7 @@
  #include "settings.h"
  #include "private.h"
  #include "missing.h"
+#include "pem-private.h"
 
  struct setting_data {
  	char *key;

ell/settings.c struct setting_data { · line 63

Quoted 6 lines
ell/settings.c +3 -0
@@ -57,6 +58,9 @@ struct setting_data {
  struct group_data {
  	char *name;
  	struct l_queue *settings;
+	/* settings and ext_data/type are mutually exclusive */
+	char *ext_type;
+	char *ext_data;

It seems like this belongs in a separate data structure. Maybe

embedded_group_data {
	char *name;
	char type[32];
	uint8_t data[0];
};
Quoted 24 lines
ell/settings.c +9 -1
  };
 
  struct l_settings {
@@ -81,7 +85,13 @@ static void group_destroy(void *data)
  	struct group_data *group = data;
 
  	l_free(group->name);
-	l_queue_destroy(group->settings, setting_destroy);
+
+	if (group->settings)
+		l_queue_destroy(group->settings, setting_destroy);
+	else {
+		l_free(group->ext_data);
+		l_free(group->ext_type);
+	}
 
  	l_free(group);
  }
@@ -222,6 +232,168 @@ static char *escape_value(const char *value)
  	return ret;
  }
 
+/*
+ * The PEM APIs deal with unsigned bytes since they ultimately end up decoding

ell/settings.c static char *escape_value(const char *value) · line 246

Quoted 10 lines
ell/settings.c +10 -0
+ * the base64 into a raw buffer. Since we are using these to just return the
+ * already encoded PEMs as textual data we are using char *. These macros just
+ * take care of the casting between the two.
+ */
+#define IS_PEM_START(buf, len, llen) \
+	(const char *)pem_is_start_boundary((const uint8_t *)buf, len, llen)
+
+#define IS_PEM_END(buf, len, label, llen) \
+	pem_is_end_boundary((const uint8_t *)buf, len, \
+				(const uint8_t *)label, label_len)

Yeah I think we screwed up the pem APIs a bit. I think that l_pem_load_buffer should likely take a const void *. And maybe pem_load_private_key should do as well. The start/end_boundary functions should probably be converted to take a const void * too.

Quoted 53 lines
ell/settings.c +53 -0
+
+static const char *next_eol(const char *data, size_t len)
+{
+	const char *eol;
+
+	for (eol = data; eol < data + len; eol++)
+		if (*eol == '\r' || *eol == '\n')
+			break;
+
+	return eol;
+}
+
+/*
+ * This was modeled after pem_load_buffer, except actually decoding the data
+ * is not done. Support for parsing multiple PEMS in a row (chain) was also
+ * added.
+ */
+static bool parse_pem(const char *data, size_t total_len, size_t *data_len)
+{
+	const char *eol;
+	const char *label;
+	size_t label_len;
+	const char *start = NULL;
+
+	while (total_len) {
+		eol = next_eol(data, total_len);
+
+		if (!start) {
+			label = IS_PEM_START(data, eol - data, &label_len);
+			if (label)
+				start = label - strlen("-----BEGIN ");
+		} else if (start && IS_PEM_END(data, eol - data,
+						label, label_len)) {
+			const char *new_eol;
+
+			new_eol = next_eol(eol + 1, total_len - (eol - data));
+
+			label = IS_PEM_START(eol + 1, new_eol - eol - 1,
+							&label_len);
+			if (!label) {
+				/* No more PEMs in chain */
+				*data_len = eol - start + 1;
+				return true;
+			}
+
+			/* Continue to next PEM */
+			eol = new_eol;
+		}
+
+		if (eol == data + total_len)
+			break;
+
+		total_len -= eol + 1 - data;

ell/settings.c static char *escape_value(const char *value) · line 309

Quoted 10 lines
ell/settings.c +10 -0
+		data = eol + 1;
+
+		if (total_len && *eol == '\r' && *data == '\n') {
+			data++;
+			total_len--;
+		}
+	}
+
+	return false;
+}

It almost seems like you want a version of 'pem_load_buffer' that simply doesn't perform the base64 decode...

ell/settings.c static char *escape_value(const char *value) · line 313

Quoted 4 lines
ell/settings.c +4 -0
+
+struct group_extension {
+	char *name;
+	bool (*parse)(const char *data, size_t total_len, size_t *data_len);

Just do ssize_t (*parse)(const void *data, size_t len);

Quoted 31 lines
ell/settings.c +31 -0
+};
+
+static struct group_extension pem_extension = {
+	.name = "pem",
+	.parse = parse_pem,
+};
+
+struct group_extension *extensions[] = {
+	&pem_extension,
+	NULL
+};
+
+static struct group_extension *find_group_extension(const char *type)
+{
+	unsigned int i;
+
+	for (i = 0; extensions[i]; i++) {
+		if (!strcmp(type, extensions[i]->name))
+			return extensions[i];
+	}
+
+	return NULL;
+}
+
+static bool parse_extended_group(struct l_settings *setting, const char *data,
+					size_t line_len, size_t total_len,
+					size_t *offset)
+{
+	struct group_data *group;
+	struct group_extension *ext;
+	char *ptr1, *ptr2;

ell/settings.c static char *escape_value(const char *value) · line 354

Quoted 10 lines
ell/settings.c +10 -0
+	char *type, *name;
+
+	/* Must be at least [@a(a)b]\n */
+	if (line_len < 7)
+		return false;
+
+	/* extended type */
+	ptr1 = strchr(data, '@');
+	if (!ptr1)
+		return false;

You sort of already checked this in the caller. And you can't assume any of these are null terminated.

Quoted 54 lines
ell/settings.c +43 -0
+
+	ptr2 = strchr(ptr1 + 1, '@');
+	if (!ptr2)
+		return false;
+
+	type = l_strndup(ptr1 + 1, ptr2 - ptr1 - 1);
+
+	ptr1 = strchr(ptr2, ']');
+	if (!ptr1)
+		goto free_type;
+
+	if (ptr1[1] != '\n')
+		goto free_type;
+
+	if (ptr1 - ptr2 - 1 < 1)
+		goto free_type;
+
+	name = l_strndup(ptr2 + 1, ptr1 - ptr2 - 1);
+
+	ext = find_group_extension(type);
+	if (!ext)
+		goto free_name;
+
+	if (!ext->parse(ptr1 + 2, total_len - line_len, offset))
+		goto free_name;
+
+	group = l_new(struct group_data, 1);
+	group->name = name;
+	group->ext_type = type;
+	group->ext_data = l_strndup(ptr1 + 2, *offset);
+
+	l_queue_push_tail(setting->groups, group);
+
+	return true;
+
+free_name:
+	l_free(name);
+free_type:
+	l_free(type);
+	return false;
+}
+
  static bool parse_group(struct l_settings *settings, const char *data,
  			size_t len, size_t line)
  {
@@ -382,6 +554,7 @@ LIB_EXPORT bool l_settings_load_from_data(struct l_settings *settings,
  	const char *eol;
  	size_t line = 1;
  	size_t line_len;
+	size_t ext_offset = 0;
 
  	if (unlikely(!settings || !data || !len))
  		return false;

ell/settings.c LIB_EXPORT bool l_settings_load_from_data(struct l_settings *settings, · line 580

Quoted 5 lines
ell/settings.c +1 -1
@@ -404,7 +577,19 @@ LIB_EXPORT bool l_settings_load_from_data(struct l_settings *settings,
 
  		line_len = eol - data - pos;
 
-		if (data[pos] == '[') {
+		if (data[pos] == '[' && data[pos + 1] == '@') {

Careful of buffer overflows

Quoted 15 lines
ell/settings.c +12 -0
+			r = parse_extended_group(settings, data + pos,
+							line_len, len - pos,
+							&ext_offset);
+			if (!r)
+				return false;
+
+			/*
+			 * This is the offset for the actual raw data, the
+			 * group line will be offset below
+			 */
+			pos += ext_offset;
+		} else if (data[pos] == '[') {
  			r = parse_group(settings, data + pos, line_len, line);
  			if (r)
  				has_group = true;

<snip>

ell/settings.h bool l_settings_remove_key(struct l_settings *settings, const char *group_name, · line 128

Quoted 11 lines
ell/settings.h +3 -0
index 0da9f55..bf7bc44 100644
@@ -123,6 +123,11 @@ bool l_settings_remove_key(struct l_settings *settings, const char *group_name,
  				const char *key);
  bool l_settings_remove_group(struct l_settings *settings,
  				const char *group_name);
+
+bool l_settings_is_extended_group(struct l_settings *settings,
+					const char *group);

I would actually keep extended groups separately. So maybe

char **l_settings_get_embedded_groups(settings); bool l_settings_has_embedded_group(settings, group);

const char *l_settings_get_embedded_value(settings, group, char **out_type);

or perhaps
const char *l_settings_get_embedded_type(settings, group);

It is arguable whether you want to set or remove embedded groups, so we can leave that capability out for now.

ell/settings.h bool l_settings_remove_key(struct l_settings *settings, const char *group_name, · line 133

Quoted 6 lines
ell/settings.h +2 -0
+const char *l_settings_get_extended_data(struct l_settings *settings,
+						const char *group);
  #ifdef __cplusplus
  }
  #endif

Regards,
-Denis

#7 lore raw

PATCH 5/5 unit/test-settings: add pem extended type tests

In reply to James Prestwood
---
 unit/settings.test   |  28 ++++++
 unit/test-settings.c | 234 ++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 259 insertions(+), 3 deletions(-)
unit/settings.test +28 -0
index 3884f7d..5f2d4f8 100644
@@ -20,3 +20,31 @@ StringList=Foo,Bar,Baz
 StringListEmpty=
 StringListOne=FooBarBaz
 StringWithSpaces=Bar B Q
+
+[@pem(a)certificate]
+-----BEGIN CERTIFICATE-----
+MIIEajCCA1KgAwIBAgIUKIOXQXEyHZsOFw/+ihDBNGTQnWUwDQYJKoZIhvcNAQEL
+BQAweDE1MDMGA1UECgwsSW50ZXJuYXRpb25hbCBVbmlvbiBvZiBFeGFtcGxlIE9y
+Z2FuaXphdGlvbnMxHzAdBgNVBAMMFkNlcnRpZmljYXRlIGlzc3VlciBndXkxHjAc
+BgkqhkiG9w0BCQEWD2NhQG1haWwuZXhhbXBsZTAeFw0xOTA5MTYxNzEyNThaFw00
+NzAyMDExNzEyNThaMHgxNTAzBgNVBAoMLEludGVybmF0aW9uYWwgVW5pb24gb2Yg
+RXhhbXBsZSBPcmdhbml6YXRpb25zMR8wHQYDVQQDDBZDZXJ0aWZpY2F0ZSBpc3N1
+ZXIgZ3V5MR4wHAYJKoZIhvcNAQkBFg9jYUBtYWlsLmV4YW1wbGUwggEiMA0GCSqG
+SIb3DQEBAQUAA4IBDwAwggEKAoIBAQCjovj3aq26sAQ0k5vD/BVp40p0lhz1+Wet
+1EcdQa1arVIca9nhfvoHfJAmK+zzqQLbvI0/e2if4X6OKf41g7w7VaYS9qv5jZZ0
+v/7aL6PUa2F7C9HG/vuIII/dRvP2uQ43PLxeTeZyj7bBUB9xCFCpzB+7AZuUuH0H
+ABaC9CAGZWImBY5NUXST7E/BsvqU80KJglDovcabthvwoekji9DC/wwISLE1e9cO
+A9IB0Co0mA1ME6wzrawmuTzxUw9BsmEhbKhFGBRwIrrq0r4GvDmeMFiZjXv+I0vq
+wSCyRtgoeBmyemqIEgiN4Z23V7ps3dbYF/tw96Zj7rd5gtjY9VSdAgMBAAGjgesw
+gegwDwYDVR0TBAgwBgEB/wIBADAdBgNVHQ4EFgQUnvllvvxy17VqqR0FfgS4NwzX
+jScwgbUGA1UdIwSBrTCBqoAUnvllvvxy17VqqR0FfgS4NwzXjSehfKR6MHgxNTAz
+BgNVBAoMLEludGVybmF0aW9uYWwgVW5pb24gb2YgRXhhbXBsZSBPcmdhbml6YXRp
+b25zMR8wHQYDVQQDDBZDZXJ0aWZpY2F0ZSBpc3N1ZXIgZ3V5MR4wHAYJKoZIhvcN
+AQkBFg9jYUBtYWlsLmV4YW1wbGWCFCiDl0FxMh2bDhcP/ooQwTRk0J1lMA0GCSqG
+SIb3DQEBCwUAA4IBAQBROAyWfQyKXQ007U6ctgihHbg/lsfEEfeNPG+QRVt8/e53
+4fH6scuY9bW7CZQSdiBo178ITHrIOo2CuFMa0ysnW3V1M9/s0dUYjBHYdpTEEQ+d
+tgm1uRLiTsYeBtueRItEmZU6JjgmvAH8i1UqI0e5iYlfnovPmftpqIwRH7k7A9kS
+SehC9QkkrnIttDEoeYTGhLOJu1Fx2cwAodce6VNgz/k1zIXY5Tprg440zrCwc+th
+MpX48F31ggg8Wd5N6Xg1nricGwL8K90ts6xvwF1WwKsg6BeYdyC0eYBqQ41MA/7P
+DK3OGM6cC5tbQGWaIT0Q407GJBGpaijDicA2YqlK
+-----END CERTIFICATE-----
unit/test-settings.c +231 -3

270 lines not on this page · raw message

Signature · 1 line
2.17.1