Skip to contentSkip to conversation

non-gcc-specific exception.sh

PATCH In review Michael Shigorin 2 participants 5 messages Oct 7, 2019 – Oct 12, 2019 SELinux security module

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 non-gcc-specific exception.sh

Hello,
please find attached the patch to (hopefully) improve self-surgery script that uses gcc-specific -aux-info now. Should help clang, icc and the like (in my case there's no proper gcc port for the target platform just yet).

Signature · 2 lines
 ---- WBR, Michael Shigorin / http://altlinux.org
  ------ http://opennet.ru / http://anna-news.info

From ccfa41adf46367e132ba7556fad53dfce9ddffaf Mon Sep 17 00:00:00 2001
From: Michael Shigorin <[email protected]>
Date: Mon, 7 Oct 2019 16:00:39 +0300
Subject: [PATCH] exception.sh: fixed to work with non-gcc

-aux-info is a gccism absent in clang, icc and the like; let's reimplement this in a slightly cleaner way with -E.

Suggested-by Evgeny Mareev
---
 libselinux/src/exception.sh  | 12 +++++-------
 libsemanage/src/exception.sh | 12 +++++-------
 2 files changed, 10 insertions(+), 14 deletions(-)
libselinux/src/exception.sh +5 -7
index d6c8c71..459295c 100755
@@ -15,10 +15,8 @@ echo "
 ;;
 esac
 }
-if ! ${CC:-gcc} -x c -c -I../include - -aux-info temp.aux < ../include/selinux/selinux.h
-then
-    # clang does not support -aux-info so fall back to gcc
-    gcc -x c -c -I../include - -aux-info temp.aux < ../include/selinux/selinux.h
-fi
-for i in `awk '/<stdin>.*extern int/ { print $6 }' temp.aux`; do except $i ; done 
-rm -f -- temp.aux -.o
+
+# equivalent of gcc-specific -aux-info
+${CC:-gcc} -E -I../include -xc ../include/selinux/selinux.h |
+	sed -n 's/^extern * int *\(\w*\) *(.*$/\1/p' |
+	while read f; do except $f; done
libsemanage/src/exception.sh +5 -7
index 97bc2ae..11586a1 100644
@@ -9,10 +9,8 @@ echo "
 }
 "
 }
-if ! ${CC:-gcc} -x c -c -I../include - -aux-info temp.aux < ../include/semanage/semanage.h
-then
-    # clang does not support -aux-info so fall back to gcc
-    gcc -x c -c -I../include - -aux-info temp.aux < ../include/semanage/semanage.h
-fi
-for i in `awk '/extern int/ { print $6 }' temp.aux`; do except $i ; done
-rm -f -- temp.aux -.o
+
+# equivalent of gcc-specific -aux-info
+${CC:-gcc} -E -I../include -xc ../include/semanage/semanage.h |
+	sed -n 's/^extern * int *\(\w*\) *(.*$/\1/p' |
+	while read f; do except $f; done
Signature · 1 line
2.10.4
#1 lore raw
Quoted 10 lines · Michael Shigorin

Hello,
please find attached the patch to (hopefully) improve self-surgery script that uses gcc-specific -aux-info now. Should help clang, icc and the like (in my case there's no proper gcc port for the target platform just yet).

--
---- WBR, Michael Shigorin / http://altlinux.org
------ http://opennet.ru / http://anna-news.info

Hello,
How did you test this patch? On my system (Arch Linux x86-64), I get the following differences in the generated list of functions, for libselinux:

+select
+pselect
-selinuxfs_exists

This is because /usr/include/sys/select.h contains "extern int select (int __nfds, fd_set *__restrict __readfds," and "extern int pselect (int __nfds, fd_set *__restrict __readfds,", and because libselinux/include/selinux/selinux.h contains "int selinuxfs_exists(void);" without "extern". Your patch therefore changes things in a way that seems unintended.

I nevertheless agree that making the build system use clang when CC=clang is a good idea. As the regular expression you sent is quite fragile, a possible way of preventing issues such as the differences I found is to try using both methods (-aux-info and -E) in a "make test" target and fail with a fatal error when they do not produce the same output (this fatal error would be caught by a continuous-integration system, which would make the developers aware of something wrong).

Moreover, please send your patch inline, if possible (for example with "git send-email"), and add a "Signed-off-by:" line as documented in
https://github.com/SELinuxProject/selinux/blob/master/CONTRIBUTING.md#contributing-code.

Thanks,
Nicolas

#2 lore raw

Review

Quoted 11 lines · Nicolas Iooss
Quoted 4 lines

please find attached the patch to (hopefully) improve self-surgery script that uses gcc-specific -aux-info now. Should help clang, icc and the like (in my case there's no proper gcc port for the target platform just yet).

How did you test this patch? On my system (Arch Linux x86-64), I get the following differences in the generated list of functions, for libselinux:

(quoted diff) +2 -1
+select
+pselect
-selinuxfs_exists

selinuxfs_exists is declared as just int, not external int, and gets filtered out with awk one-liner within the original script.

I've spotted select/pselect and have paid some attention to zeroing the diff *but* overlooked the <stdin> filter in the script from libselinux-2.9 tagged tree I started looking at. Evgeny suggested doing something about the changed source marker format either, shame on me for distracting.

Quoted 6 lines

This is because /usr/include/sys/select.h contains "extern int select (int __nfds, fd_set *__restrict __readfds," and "extern int pselect (int __nfds, fd_set *__restrict __readfds,", and because libselinux/include/selinux/selinux.h contains "int selinuxfs_exists(void);" without "extern". Your patch therefore changes things in a way that seems unintended.

Will this one do better? Looks a bit messy though[1]:

gcc -E -I../include -xc ../include/selinux/selinux.h |

  awk -F '[ (]' '/^#/{p=0} \
    /^# .*selinux\/selinux\.h/{p=1} \
    /^extern int/{if(p==1){print $3}}'
Quoted 7 lines

As the regular expression you sent is quite fragile a possible way of preventing issues such as the differences I found is to try using both methods (-aux-info and -E) in a "make test" target and fail with a fatal error when they do not produce the same output (this fatal error would be caught by a continuous-integration system, which would make the developers aware of something wrong).

Good test for those who have it handy, my primary intent was to be able to at least build without (not-yet-ported) gcc.

Quoted 3 lines

Moreover, please send your patch inline, if possible (for example with "git send-email"), and add a "Signed-off-by:" line as documented in
https://github.com/SELinuxProject/selinux/blob/master/CONTRIBUTING.md#contributing-code.

Thank you for the review and suggestions, please have a look at the proposed handler replacement; if it's ok I'll arrange it as a commit and hopefully figure out the test tomorrow, getting sleepy by now...

---

[1] start/end patterns don't cut it for me given repetitions:

sed -n '/^# .*selinux\/selinux\.h/,/^#/{s/^extern int \(\w*\) *(.*$/\1/p}'

skips e.g. security_deny_unknown given gcc's output but works fine with somewhat differing lcc's one.

Signature · 2 lines
 ---- WBR, Michael Shigorin / http://altlinux.org
  ------ http://opennet.ru / http://anna-news.info
#3 lore raw
Quoted 48 lines · Michael Shigorin

Review

Quoted 11 lines · Nicolas Iooss
Quoted 4 lines

please find attached the patch to (hopefully) improve self-surgery script that uses gcc-specific -aux-info now. Should help clang, icc and the like (in my case there's no proper gcc port for the target platform just yet).

How did you test this patch? On my system (Arch Linux x86-64), I get the following differences in the generated list of functions, for libselinux:

(quoted diff) +2 -1
+select
+pselect
-selinuxfs_exists

selinuxfs_exists is declared as just int, not external int, and gets filtered out with awk one-liner within the original script.

I've spotted select/pselect and have paid some attention to zeroing the diff *but* overlooked the <stdin> filter in the script from libselinux-2.9 tagged tree I started looking at. Evgeny suggested doing something about the changed source marker format either, shame on me for distracting.

Quoted 6 lines

This is because /usr/include/sys/select.h contains "extern int select (int __nfds, fd_set *__restrict __readfds," and "extern int pselect (int __nfds, fd_set *__restrict __readfds,", and because libselinux/include/selinux/selinux.h contains "int selinuxfs_exists(void);" without "extern". Your patch therefore changes things in a way that seems unintended.

Will this one do better? Looks a bit messy though[1]:

gcc -E -I../include -xc ../include/selinux/selinux.h |

  awk -F '[ (]' '/^#/{p=0} \
    /^# .*selinux\/selinux\.h/{p=1} \
    /^extern int/{if(p==1){print $3}}'
Quoted 7 lines

As the regular expression you sent is quite fragile a possible way of preventing issues such as the differences I found is to try using both methods (-aux-info and -E) in a "make test" target and fail with a fatal error when they do not produce the same output (this fatal error would be caught by a continuous-integration system, which would make the developers aware of something wrong).

Good test for those who have it handy, my primary intent was to be able to at least build without (not-yet-ported) gcc.

Quoted 8 lines
Quoted 3 lines

Moreover, please send your patch inline, if possible (for example with "git send-email"), and add a "Signed-off-by:" line as documented in
https://github.com/SELinuxProject/selinux/blob/master/CONTRIBUTING.md#contributing-code.

Thank you for the review and suggestions, please have a look at the proposed handler replacement; if it's ok I'll arrange it as a commit and hopefully figure out the test tomorrow, getting sleepy by now...

Hello,
I wanted to spend some time testing different approaches before replying. Your awk command works fine, but I am wondering whether its complexity is needed at all. For example, why not directly grab the "extern int" lines in selinux.h (using sed -n s/.../.../p). This would work well with libselinux and avoid using awk for this simple task, but not for libsemanage... or it could, with:

    cat ../include/semanage/*.h | \
    sed -n 's/^extern \+int \+\([0-9A-Za-z_]\+\) *(.*$/\1/p'

Actually, the main reason why it is currently not possible to migrate from "gcc -aux-info and awk" to "cat | sed" is caused by the fact that the file generated by "gcc -aux-info" adds "extern" before all function prototypes. Adding "extern" directly to the prototypes of all exported functions in header files seems to be required in order to use something else than "gcc -aux-info" to generate the glue code that handles exception in Python bindings.

Therefore I suggest the following approach:
1. add "extern" to all exported functions in libselinux and libsemanage public headers (I can send a patch for this tomorrow if you don't want to do this)
2. change exception.sh to use something else than "gcc -aux-info" to generate the glue code
3. add something so that "make test" compares the result of "$CC -aux-info" (if $CC is gcc-compatible) with the new way to find functions, in order to report differences in the generated glue code. This could for example consist in adding some code in exception.sh such that "./exception.sh test" performs this check.

Would this suit you?

By the way, I found out that libsemanage generates glue code for select() and pselect() too, as it does not filter the output of "gcc -aux-info". This is a bug that needs to be fixed.

Thanks,
Nicolas

#4 lore raw
Quoted 9 lines · Nicolas Iooss
Quoted 6 lines

Will this one do better? Looks a bit messy though[1]:

gcc -E -I../include -xc ../include/selinux/selinux.h |

  awk -F '[ (]' '/^#/{p=0} \
    /^# .*selinux\/selinux\.h/{p=1} \
    /^extern int/{if(p==1){print $3}}'

I wanted to spend some time testing different approaches before replying. Your awk command works fine, but I am wondering whether its complexity is needed at all.

I've considered this approach too but decided to try and mimic what's done already first, and maybe try the next step afterwards (albeit "a temporary solution is the most permanent one" is a well known effect indeed :-).

Quoted 8 lines

For example, why not directly grab the "extern int" lines in selinux.h (using sed -n s/.../.../p). This would work well with libselinux and avoid using awk for this simple task, but not for libsemanage... or it could, with:

    cat ../include/semanage/*.h | \
    sed -n 's/^extern \+int \+\([0-9A-Za-z_]\+\) *(.*$/\1/p'
Quoted 7 lines

Actually, the main reason why it is currently not possible to migrate from "gcc -aux-info and awk" to "cat | sed" is caused by the fact that the file generated by "gcc -aux-info" adds "extern" before all function prototypes. Adding "extern" directly to the prototypes of all exported functions in header files seems to be required in order to use something else than "gcc -aux-info" to generate the glue code that handles exception in Python bindings.

That's another part I'm not strong at, being rather shell/make sysadmin/maintainer type than a proper C developer; only spotted the selinuxfs_exists prototype difference, in particular.

Quoted 4 lines

Therefore I suggest the following approach:
1. add "extern" to all exported functions in libselinux and libsemanage public headers (I can send a patch for this tomorrow if you don't want to do this)

I'd be grateful to you as touching code I have no clue about is something I try to avoid.

Quoted 2 lines

2. change exception.sh to use something else than "gcc -aux-info" to generate the glue code

This is what I can take care of, -E way or through direct parsing.

Quoted 5 lines

3. add something so that "make test" compares the result of "$CC -aux-info" (if $CC is gcc-compatible) with the new way to find functions, in order to report differences in the generated glue code. This could for example consist in adding some code in exception.sh such that "./exception.sh test" performs this check.

I can do that either given your support with review.

Quoted 1 line

Would this suit you?

Definitely.

Quoted 4 lines

By the way, I found out that libsemanage generates glue code for select() and pselect() too, as it does not filter the output of "gcc -aux-info". This is a bug that needs to be fixed.

~/git/libselinux/libsemanage/src> touch FILTER-OUT-p_select-TOO

Signature · 2 lines
 ---- WBR, Michael Shigorin / http://altlinux.org
  ------ http://opennet.ru / http://anna-news.info