Skip to contentSkip to conversation

Lazy config init in libselinux

PATCH In review Steve G 2 participants 11 messages Feb 26, 2007 – Feb 27, 2007 SELinux security module

lore
Share & tools
lore

Steve G

author
#0 lore raw

PATCH 1/1 Lazy config init in libselinux

Hi,

After running strace a number of times in the other performance patch, I realized that we are reading a config file in a lot of cases where we don't even use the results. Example, "ls" opens, reads, and parses /etc/selinux/config and it doesn't care unless you pass the -Z flag. So...this patch does 2 things. It does a lazy read of the config file and it moves the check for /etc/security to be a second class citizen instead of something checked for first. This patch should make shell scripts run faster.

____________________________________________________________________________________
Bored stiff? Loosen up...
Download and play hundreds of games for free on Yahoo! Games.
http://games.yahoo.com/games/front

src/selinux_config.c +34 -13
diff -urp libselinux-2.0.4.orig/src/selinux_config.c libselinux-2.0.4/src/selinux_config.c
@@ -7,6 +7,7 @@
 #include <stdlib.h>
 #include <limits.h>
 #include <unistd.h>
+#include <errno.h>
 #include "selinux_internal.h"
 #include "get_default_type_internal.h"
 
@@ -92,6 +93,9 @@ static const uint16_t compat_file_path_i
 #undef L2
 
 static int use_compat_file_path;
+static int init_selinux_config_done;
+static int init_selinux_config(void);
+
 
 int selinux_getenforcemode(int *enforce)
 {
@@ -144,6 +148,10 @@ static char *selinux_policytype;
 
 int selinux_getpolicytype(char **type)
 {
+	if (!init_selinux_config_done) {
+		if (init_selinux_config() < 0)
+			return -1;
+	}
 	if (!selinux_policytype)
 		return -1;
 	*type = strdup(selinux_policytype);
@@ -155,9 +163,8 @@ hidden_def(selinux_getpolicytype)
 static char *selinux_policyroot = NULL;
 static char *selinux_rootpath = NULL;
 
-static void init_selinux_config(void) __attribute__ ((constructor));
 
-static void init_selinux_config(void)
+static int init_selinux_config(void)
 {
 	int i, *intptr;
 	size_t line_len;
@@ -166,13 +173,7 @@ static void init_selinux_config(void)
 	FILE *fp;
 
 	if (selinux_policyroot)
-		return;
-	if (access(SELINUXDIR, F_OK) != 0) {
-		selinux_policyroot = SECURITYDIR;
-		selinux_rootpath = SECURITYDIR;
-		use_compat_file_path = 1;
-		return;
-	}
+		return 0;
 
 	selinux_rootpath = SELINUXDIR;
 	fp = fopen(SELINUXCONFIG, "r");
@@ -192,7 +193,7 @@ static void init_selinux_config(void)
 				selinux_policytype = type =
 				    strdup(buf_p + sizeof(SELINUXTYPETAG) - 1);
 				if (!type)
-					return;
+					return -1;
 				end = type + strlen(type) - 1;
 				while ((end > type) &&
 				       (isspace(*end) || iscntrl(*end))) {
@@ -226,16 +227,22 @@ static void init_selinux_config(void)
 		}
 		free(line_buf);
 		fclose(fp);
+	} else if (errno == ENOENT && access(SECURITYDIR, F_OK) == 0) {
+		selinux_policyroot = SECURITYDIR;
+		selinux_rootpath = SECURITYDIR;
+		use_compat_file_path = 1;
+		init_selinux_config_done = 1;
+		return 0;
 	}
 
 	if (!type) {
 		selinux_policytype = type = strdup(SELINUXDEFAULT);
 		if (!type)
-			return;
+			return -1;
 	}
 
 	if (asprintf(&selinux_policyroot, "%s%s", SELINUXDIR, type) == -1)
-		return;
+		return -1;
 
 	for (i = 0; i < NEL; i++)
 		if (asprintf(&file_paths[i], "%s%s",
@@ -243,8 +250,10 @@ static void init_selinux_config(void)
 			     file_path_suffixes_data.str +
 			     file_path_suffixes_idx[i])
 		    == -1)
-			return;
+			return -1;
 	use_compat_file_path = 0;
+	init_selinux_config_done = 1;
+	return 0;
 }
 
 static void fini_selinux_policyroot(void) __attribute__ ((destructor));
@@ -268,6 +277,10 @@ static void fini_selinux_policyroot(void
 
 static const char *get_path(int idx)
 {
+	if (!init_selinux_config_done) {
+		if (init_selinux_config() < 0)
+			return NULL;
+	}
 	if (!use_compat_file_path)
 		return file_paths[idx];
 
@@ -283,11 +296,19 @@ hidden_def(selinux_default_type_path)
 
 const char *selinux_policy_root()
 {
+	if (!init_selinux_config_done) {
+		if (init_selinux_config() < 0)
+			return NULL;
+	}
 	return selinux_policyroot;
 }
 
 const char *selinux_path()
 {
+	if (!init_selinux_config_done) {
+		if (init_selinux_config() < 0)
+			return NULL;
+	}
 	return selinux_rootpath;
 }
 
#1 lore raw
Quoted 11 lines · Steve G

Hi,

After running strace a number of times in the other performance patch, I realized that we are reading a config file in a lot of cases where we don't even use the results. Example, "ls" opens, reads, and parses /etc/selinux/config and it doesn't care unless you pass the -Z flag. So...this patch does 2 things. It does a lazy read of the config file and it moves the check for /etc/security to be a second class citizen instead of something checked for first. This patch should make shell scripts run faster.

The problem with doing it on first use is it isn't thread-safe; the code is initializing globals for the library's subsequent use. And making those variables per-thread would be wasteful as they truly are global state to the library. So initializing them in the library constructor is the right thing to do.

Signature · 8 lines
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to [email protected] with
the words "unsubscribe selinux" without quotes as the message.
author
#2 lore raw
Quoted 6 lines

The problem with doing it on first use is it isn't thread-safe; the code is initializing globals for the library's subsequent use. And making those variables per-thread would be wasteful as they truly are global state to the library. So initializing them in the library constructor is the right thing to do.

How do you feel about __libc_once ?

-Steve

____________________________________________________________________________________
Need a quick answer? Get one in minutes from people who know. Ask your question on www.Answers.yahoo.com

--
This message was distributed to subscribers of the selinux mailing list. If you no longer wish to subscribe, send mail to [email protected] with the words "unsubscribe selinux" without quotes as the message.

#3 lore raw
Quoted 7 lines · Steve G
Quoted 5 lines

The problem with doing it on first use is it isn't thread-safe; the code is initializing globals for the library's subsequent use. And making those variables per-thread would be wasteful as they truly are global state to the library. So initializing them in the library constructor is the right thing to do.

How do you feel about __libc_once ?

Not familiar with it - enlighten me. How does it differ from a constructor? How does it work?

Signature · 8 lines
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to [email protected] with
the words "unsubscribe selinux" without quotes as the message.
author
#4 lore raw
Quoted 2 lines

Not familiar with it - enlighten me. How does it differ from a constructor? How does it work?

Its a wrapper around pthread_once. pthread_once gurantees that something gets executed only one time; IOW it can be used as an on-demand constructor. libselinux does not need to link against pthreads, so that does not change. Here's the tested patch.

I added the following audit rule "-w /etc/selinux/config -k selinux-config". Then rebooted and run "ausearch --start recent -k selinux-config --raw | aureport -executable --summary" to see which programs were accessing selinux config files. You might have to adjust your start time instead of using "recent". But anyways it does cut the use. The speedup varies with the work load. I think things that are heavy in ls, cp, mv, mount, id, find, etc are likely to have the most benefit.

-Steve

____________________________________________________________________________________
Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com

src/selinux_config.c +17 -7
diff -urp libselinux-2.0.4.orig/src/selinux_config.c libselinux-2.0.4/src/selinux_config.c
@@ -7,8 +7,10 @@
 #include <stdlib.h>
 #include <limits.h>
 #include <unistd.h>
+#include <errno.h>
 #include "selinux_internal.h"
 #include "get_default_type_internal.h"
+#include <bits/libc-lock.h>
 
 #define SELINUXDIR "/etc/selinux/"
 #define SELINUXCONFIG SELINUXDIR "config"
@@ -93,6 +95,11 @@ static const uint16_t compat_file_path_i
 
 static int use_compat_file_path;
 
+/* Protect the configuration variables */
+__libc_once_define(static, once);
+static void init_selinux_config(void);
+
+
 int selinux_getenforcemode(int *enforce)
 {
 	int ret = -1;
@@ -144,6 +151,8 @@ static char *selinux_policytype;
 
 int selinux_getpolicytype(char **type)
 {
+	__libc_once(once, init_selinux_config);
+
 	if (!selinux_policytype)
 		return -1;
 	*type = strdup(selinux_policytype);
@@ -155,7 +164,6 @@ hidden_def(selinux_getpolicytype)
 static char *selinux_policyroot = NULL;
 static char *selinux_rootpath = NULL;
 
-static void init_selinux_config(void) __attribute__ ((constructor));
 
 static void init_selinux_config(void)
 {
@@ -167,12 +175,6 @@ static void init_selinux_config(void)
 
 	if (selinux_policyroot)
 		return;
-	if (access(SELINUXDIR, F_OK) != 0) {
-		selinux_policyroot = SECURITYDIR;
-		selinux_rootpath = SECURITYDIR;
-		use_compat_file_path = 1;
-		return;
-	}
 
 	selinux_rootpath = SELINUXDIR;
 	fp = fopen(SELINUXCONFIG, "r");
@@ -226,6 +228,11 @@ static void init_selinux_config(void)
 		}
 		free(line_buf);
 		fclose(fp);
+	} else if (errno == ENOENT && access(SECURITYDIR, F_OK) == 0) {
+		selinux_policyroot = SECURITYDIR;
+		selinux_rootpath = SECURITYDIR;
+		use_compat_file_path = 1;
+		return;
 	}
 
 	if (!type) {
@@ -268,6 +275,7 @@ static void fini_selinux_policyroot(void
 
 static const char *get_path(int idx)
 {
+	__libc_once(once, init_selinux_config);
 	if (!use_compat_file_path)
 		return file_paths[idx];
 
@@ -283,11 +291,13 @@ hidden_def(selinux_default_type_path)
 
 const char *selinux_policy_root()
 {
+	__libc_once(once, init_selinux_config);
 	return selinux_policyroot;
 }
 
 const char *selinux_path()
 {
+	__libc_once(once, init_selinux_config);
 	return selinux_rootpath;
 }
 
#5 lore raw
Quoted 8 lines · Steve G
Quoted 2 lines

Not familiar with it - enlighten me. How does it differ from a constructor? How does it work?

Its a wrapper around pthread_once. pthread_once gurantees that something gets executed only one time; IOW it can be used as an on-demand constructor. libselinux does not need to link against pthreads, so that does not change. Here's the tested patch.

Quoted 7 lines

I added the following audit rule "-w /etc/selinux/config -k selinux-config". Then rebooted and run "ausearch --start recent -k selinux-config --raw | aureport -executable --summary" to see which programs were accessing selinux config files. You might have to adjust your start time instead of using "recent". But anyways it does cut the use. The speedup varies with the work load. I think things that are heavy in ls, cp, mv, mount, id, find, etc are likely to have the most benefit.

Interesting. A few concerns:
- I'm not clear that __libc_once is intended to be a stable API/ABI provided by glibc, given its naming convention, lack of man page, and comment at the beginning of bits/libc-lock.h. I sent Ulrich a note separately asking his opinion on using it in libselinux.
- I'm not clear on what overhead this imposes upon the functions that need to be guarded by __libc_once, and what the tradeoff is for real workloads.
- We'd have to be careful to ensure that all functions that use any global data initialized by init_selinux_config() are guarded by __libc_once(), both for the current set of global data and any future extensions. With the current set, in addition to the path functions, we'd have to ensure that the various flags set by init_selinux_config() aren't used before initialization anywhere. We can certainly audit all uses now and add such guards, but I'm a little concerned about the maintainability of this approach long term.

Signature · 8 lines
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to [email protected] with
the words "unsubscribe selinux" without quotes as the message.
author
#6 lore raw
Quoted 3 lines

- I'm not clear that __libc_once is intended to be a stable API/ABI provided by glibc,

We could copy it into a private header. Its a convenience wrapper around pthread_once, so moving it to a private header would guarantee its stability.

Quoted 3 lines

- I'm not clear on what overhead this imposes upon the functions that need to be guarded by __libc_once, and what the tradeoff is for real workloads.

I have to ask how often something really needs to know the setting of targeted or strict that is performance critical. I'd rather see the development tools or rarely used utilities take the hit in favor of runtime performance of everything else. There are far more users of "ls" than semodule.

Quoted 3 lines

- We'd have to be careful to ensure that all functions that use any global data initialized by init_selinux_config() are guarded by __libc_once(),

That is the case, today, with the patch.

Quoted 1 line

and any future extensions.

That's the trick. We can put some note at the top of the file to remind us. And some comments in the source, too.

Quoted 3 lines

With the current set, in addition to the path functions, we'd have to ensure that the various flags set by init_selinux_config() aren't used before initialization anywhere.

Sure. I think we are covered for today's file.

Quoted 2 lines

We can certainly audit all uses now and add such guards, but I'm a little concerned about the maintainability of this approach long term.

I did the audit yesterday to make sure everything was covered. Please double check me since this is important. During the audit, I was also wondering about /etc/security and whether that should die now. Moving the macro into a private header should aleviate external dependencies.

My testing of selinuxenabled with this patch and the statfs patch shows its now 21% faster. ls of an empty directory is 166 syscalls before and 147 syscalls after the patches. So that is also around 11%.

-Steve

____________________________________________________________________________________
Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com

--
This message was distributed to subscribers of the selinux mailing list. If you no longer wish to subscribe, send mail to [email protected] with the words "unsubscribe selinux" without quotes as the message.

#7 lore raw
Quoted 5 lines · Steve G
Quoted 2 lines

- I'm not clear that __libc_once is intended to be a stable API/ABI provided by glibc,

We could copy it into a private header. Its a convenience wrapper around pthread_once, so moving it to a private header would guarantee its stability.

I'm a bit confused then - why doesn't that introduce a dependency on libpthread?

Quoted 8 lines
Quoted 3 lines

- I'm not clear on what overhead this imposes upon the functions that need to be guarded by __libc_once, and what the tradeoff is for real workloads.

I have to ask how often something really needs to know the setting of targeted or strict that is performance critical. I'd rather see the development tools or rarely used utilities take the hit in favor of runtime performance of everything else. There are far more users of "ls" than semodule.

If it is only "development tools or rarely used utilities", then yes. But I think we need to assess whether that is the case - I don't think we know yet what the overall impact is.

Quoted 7 lines
Quoted 2 lines

We can certainly audit all uses now and add such guards, but I'm a little concerned about the maintainability of this approach long term.

I did the audit yesterday to make sure everything was covered. Please double check me since this is important. During the audit, I was also wondering about /etc/security and whether that should die now. Moving the macro into a private header should aleviate external dependencies.

init_selinux_config() also sets several flags based
on /etc/selinux/config that are used by other functions, and you don't appear to have explicitly added guards for them, although some end up hitting a guard due to prior call to a path function. The one that looks particularly worrisome is the cache_trans flag, used by the context translation functions, which in turn are called pervasively to translate contexts when they are passed into or out of libselinux. If we add a guard to those functions, we'll essentially hit the guard on almost every libselinux function call, just not is_selinux_enabled().

I assume too that __libc_once() or its equivalent is going to have different impact depending on whether the application is actually multi-threaded or not, so it would be interesting to see what impact it has in the multi-threaded case, even if that is less common.

I'm not trying to block performance improvement, but we do need to first understand the implications of the change, in terms of overall performance impact, correctness, and maintainability. That's all.

Signature · 8 lines
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to [email protected] with
the words "unsubscribe selinux" without quotes as the message.
author
#8 lore raw
Quoted 3 lines

I'm a bit confused then - why doesn't that introduce a dependency on libpthread?

I think I figured it out. First you have this:

# pragma weak __pthread_once

Which will make the linker ignore the function if its not found at runtime. Then we have this:

/* Call handler iff the first call. */ #define __libc_once(ONCE_CONTROL, INIT_FUNCTION) \

  do {                                                                        \
    if (__pthread_once != NULL)                                               \
      __pthread_once (&(ONCE_CONTROL), (INIT_FUNCTION));                      \
    else if ((ONCE_CONTROL) == PTHREAD_ONCE_INIT) {                           \
      INIT_FUNCTION ();                                                       \
      (ONCE_CONTROL) = 2;                                                     \
    }                                                                         \

} while (0)

So, in single thread mode the "else if" is executed. When pthreads exists it calls the function.

Quoted 2 lines

But I think we need to assess whether that is the case - I don't think we know yet what the overall impact is.

In the non-threaded mode, it doesn't look like a big hit. If __pthread_once is not defined, there's nothing that can race. If it is defined, which is rare, it will be a call to that function which Uli says is fast.

Quoted 6 lines
Quoted 2 lines

I did the audit yesterday to make sure everything was covered. Please double check me since this is important.

init_selinux_config() also sets several flags based on /etc/selinux/config that are used by other functions, and you don't appear to have explicitly added guards for them,

OK, I see 3 new variables...however, they are not used in any selinux config file I've ever seen. Are they brand new and never used? Is there a reason they are not in /etc/selinux/config by default?

Quoted 6 lines

although some end up hitting a guard due to prior call to a path function. The one that looks particularly worrisome is the cache_trans flag, used by the context translation functions, which in turn are called pervasively to translate contexts when they are passed into or out of libselinux. If we add a guard to those functions, we'll essentially hit the guard on almost every libselinux function call, just not is_selinux_enabled().

But so far no config file defines it. Is it really used? Right now everyone is getting the default value.

Quoted 3 lines

I'm not trying to block performance improvement, but we do need to first understand the implications of the change, in terms of overall performance impact, correctness, and maintainability.

Sure.

What about /etc/security...is that still used? Should that go away?

-Steve

____________________________________________________________________________________
Expecting? Get great news right away with email Auto-Check. Try the Yahoo! Mail Beta.
http://advision.webevents.yahoo.com/mailbeta/newmail_tools.html

--
This message was distributed to subscribers of the selinux mailing list. If you no longer wish to subscribe, send mail to [email protected] with the words "unsubscribe selinux" without quotes as the message.

#9 lore raw
Quoted 7 lines · Steve G
Quoted 3 lines

init_selinux_config() also sets several flags based on /etc/selinux/config that are used by other functions, and you don't appear to have explicitly added guards for them,

OK, I see 3 new variables...however, they are not used in any selinux config file I've ever seen. Are they brand new and never used? Is there a reason they are not in /etc/selinux/config by default?

They aren't new, and SETLOCALDEFS=0 is in widespread use; you should have it in your /etc/selinux/config if you are running anything newer than Fedora Core 4. But I'm not worried about the first two, because they aren't on critical paths and can be easily guarded (and happen to already hit a guard due to a prior call to a path function). Only CACHETRANS appears to be an issue. You only need to define them in /etc/selinux/config if you want something other than the default value.

Quoted 2 lines

But so far no config file defines it. Is it really used? Right now everyone is getting the default value.

It was added back when the libselinux client-side support for context translation was merged, upon someone's specific request on list. So we'd have to go back to that discussion.

Quoted 1 line

What about /etc/security...is that still used? Should that go away?

Likely obsolete, although we'd need to post an rfc asking whether any distro is still using that as the location for their policy files.

Signature · 8 lines
Stephen Smalley
National Security Agency


--
This message was distributed to subscribers of the selinux mailing list.
If you no longer wish to subscribe, send mail to [email protected] with
the words "unsubscribe selinux" without quotes as the message.
author
#10 lore raw
Quoted 8 lines
Quoted 4 lines

however, they are not used in any selinux config file I've ever seen. Are they brand new and never used? Is there a reason they are not in /etc/selinux/config by default?

Only CACHETRANS appears to be an issue. You only need to define them in /etc/selinux/config if you want something other than the default value.

I don't see that one in the config file so I think everyone is using the default value.

Quoted 3 lines

It was added back when the libselinux client-side support for context translation was merged, upon someone's specific request on list. So we'd have to go back to that discussion.

I bet it was worked around and not needed. We should revisit that discussion since it appears to not be defined in the default config file.

Quoted 4 lines
Quoted 1 line

What about /etc/security...is that still used? Should that go away?

Likely obsolete, although we'd need to post an rfc asking whether any distro is still using that as the location for their policy files.

Sure. I was just pointing out the fact its there and is likely deprecated.

-Steve

____________________________________________________________________________________
Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com

--
This message was distributed to subscribers of the selinux mailing list. If you no longer wish to subscribe, send mail to [email protected] with the words "unsubscribe selinux" without quotes as the message.