selinux testsuite

Just a little note to announce that the selinux testsuite is now up as a git tree at kernel.org. You can fetch it using

git clone git://git.kernel.org/pub/scm/tests/selinux-testsuite

The README shows how to run.

Many thanks to Stephen for keeping it running so nicely and allowing me to post it. I intend to test against Fedora regularly-ish, and to start seeing how best to nudge it to work against Debian and Ubuntu in the next month or so.

-serge

--
This message was distributed to subscribers of the selinux mailing list. If you no longer wish to subscribe, send mail to [email protected] with the words "unsubscribe selinux" without quotes as the message.

Hi Serge,

Quoted 4 lines · Serge E. Hallyn

Just a little note to announce that the selinux testsuite is now up as a git tree at kernel.org. You can fetch it using

git clone git://git.kernel.org/pub/scm/tests/selinux-testsuite

Great. So, how far does it differ from that we have in LTP ?
http://ltp.git.sourceforge.net/git/gitweb.cgi?p=ltp/ltp-dev.git;a=tree;f=testcases/kernel/security/selinux-testsuite;h=066df4cdf2f8a80a0045e338b4bb2bf0f5d37091;hb=HEAD

As you, Stephen and others in SELinux <[email protected]> were the main contributors to the SELinux test(s) in LTP, i guess the next/future development for SELinux tests will be happening in the new tree.

So, i would rather keep pulling to LTP from your tree. Now, could you please let me know:

1. Whether the new tree contains all scenarios which is present in LTP. In such a case a complete pruning of LTP SELinux tests can be done and replaced with your tree contents,
2. If some of the LTP tests are not there, then i would like to retain them in LTP side-by-side your tree contents,

Quoted 15 lines · Subrata Modak

Hi Serge,

Quoted 4 lines · Serge E. Hallyn

Just a little note to announce that the selinux testsuite is now up as a git tree at kernel.org. You can fetch it using

git clone git://git.kernel.org/pub/scm/tests/selinux-testsuite

Great. So, how far does it differ from that we have in LTP ?
http://ltp.git.sourceforge.net/git/gitweb.cgi?p=ltp/ltp-dev.git;a=tree;f=testcases/kernel/security/selinux-testsuite;h=066df4cdf2f8a80a0045e338b4bb2bf0f5d37091;hb=HEAD

As you, Stephen and others in SELinux <[email protected]> were the main contributors to the SELinux test(s) in LTP, i guess the next/future development for SELinux tests will be happening in the new tree.

Quoted 8 lines

So, i would rather keep pulling to LTP from your tree. Now, could you please let me know:

1. Whether the new tree contains all scenarios which is present in LTP. In such a case a complete pruning of LTP SELinux tests can be done and replaced with your tree contents,
2. If some of the LTP tests are not there, then i would like to retain them in LTP side-by-side your tree contents,

The new tree contains all of the selinux tests present in the LTP. You cannot however simply replace the ltp version of the tests with this tree, as this tree is a standalone testsuite and will not run within the ltp test harness. This testsuite is based on the original standalone selinux testsuite that was contributed by us to IBM to port to the LTP. We have internally maintained this testsuite in parallel to the ltp version as we have found it easier to set up, use, debug, and maintain.

Quoted 25 lines · Stephen Smalley

Quoted 15 lines · Subrata Modak

Hi Serge,

Quoted 4 lines · Serge E. Hallyn

Just a little note to announce that the selinux testsuite is now up as a git tree at kernel.org. You can fetch it using

git clone git://git.kernel.org/pub/scm/tests/selinux-testsuite

Great. So, how far does it differ from that we have in LTP ?
http://ltp.git.sourceforge.net/git/gitweb.cgi?p=ltp/ltp-dev.git;a=tree;f=testcases/kernel/security/selinux-testsuite;h=066df4cdf2f8a80a0045e338b4bb2bf0f5d37091;hb=HEAD

As you, Stephen and others in SELinux <[email protected]> were the main contributors to the SELinux test(s) in LTP, i guess the next/future development for SELinux tests will be happening in the new tree.

Quoted 8 lines

So, i would rather keep pulling to LTP from your tree. Now, could you please let me know:

1. Whether the new tree contains all scenarios which is present in LTP. In such a case a complete pruning of LTP SELinux tests can be done and replaced with your tree contents,
2. If some of the LTP tests are not there, then i would like to retain them in LTP side-by-side your tree contents,

Quoted 7 lines

The new tree contains all of the selinux tests present in the LTP. You cannot however simply replace the ltp version of the tests with this tree, as this tree is a standalone testsuite and will not run within the ltp test harness. This testsuite is based on the original standalone selinux testsuite that was contributed by us to IBM to port to the LTP. We have internally maintained this testsuite in parallel to the ltp version as we have found it easier to set up, use, debug, and maintain.

Thanks Stephen. Since you have maintained the Original-One Internally and have simultaneously contributed changes to the LTP-version of SELinux, i am depending on you/Serge/SELinux-Mailing-List-members to keep sending patches to LTP to update the LTP-version at regular intervals (off-course at your convenience ;-))

Quoted 34 lines · Subrata Modak

Quoted 25 lines · Stephen Smalley

Quoted 15 lines · Subrata Modak

Hi Serge,

Quoted 4 lines · Serge E. Hallyn

Just a little note to announce that the selinux testsuite is now up as a git tree at kernel.org. You can fetch it using

git clone git://git.kernel.org/pub/scm/tests/selinux-testsuite

Great. So, how far does it differ from that we have in LTP ?
http://ltp.git.sourceforge.net/git/gitweb.cgi?p=ltp/ltp-dev.git;a=tree;f=testcases/kernel/security/selinux-testsuite;h=066df4cdf2f8a80a0045e338b4bb2bf0f5d37091;hb=HEAD

As you, Stephen and others in SELinux <[email protected]> were the main contributors to the SELinux test(s) in LTP, i guess the next/future development for SELinux tests will be happening in the new tree.

Quoted 8 lines

So, i would rather keep pulling to LTP from your tree. Now, could you please let me know:

1. Whether the new tree contains all scenarios which is present in LTP. In such a case a complete pruning of LTP SELinux tests can be done and replaced with your tree contents,
2. If some of the LTP tests are not there, then i would like to retain them in LTP side-by-side your tree contents,

Quoted 7 lines

The new tree contains all of the selinux tests present in the LTP. You cannot however simply replace the ltp version of the tests with this tree, as this tree is a standalone testsuite and will not run within the ltp test harness. This testsuite is based on the original standalone selinux testsuite that was contributed by us to IBM to port to the LTP. We have internally maintained this testsuite in parallel to the ltp version as we have found it easier to set up, use, debug, and maintain.

Quoted 5 lines

Thanks Stephen. Since you have maintained the Original-One Internally and have simultaneously contributed changes to the LTP-version of SELinux, i am depending on you/Serge/SELinux-Mailing-List-members to keep sending patches to LTP to update the LTP-version at regular intervals (off-course at your convenience ;-))

I think that will depend on whether there is some benefit to maintaining the ltp version. We might derive some benefit if the ltp selinux testsuite were better integrated (e.g. tests run by default if SELinux is enabled on the host) and if the tests got some regular attention from the ltp maintainers. Otherwise, it may be better to just remove the tests from the ltp and point people to the standalone version.

Quoted 41 lines · Stephen Smalley

Quoted 34 lines · Subrata Modak

Quoted 25 lines · Stephen Smalley

Quoted 15 lines · Subrata Modak

Hi Serge,

Quoted 4 lines · Serge E. Hallyn

Just a little note to announce that the selinux testsuite is now up as a git tree at kernel.org. You can fetch it using

git clone git://git.kernel.org/pub/scm/tests/selinux-testsuite

Great. So, how far does it differ from that we have in LTP ?
http://ltp.git.sourceforge.net/git/gitweb.cgi?p=ltp/ltp-dev.git;a=tree;f=testcases/kernel/security/selinux-testsuite;h=066df4cdf2f8a80a0045e338b4bb2bf0f5d37091;hb=HEAD

As you, Stephen and others in SELinux <[email protected]> were the main contributors to the SELinux test(s) in LTP, i guess the next/future development for SELinux tests will be happening in the new tree.

Quoted 8 lines

So, i would rather keep pulling to LTP from your tree. Now, could you please let me know:

1. Whether the new tree contains all scenarios which is present in LTP. In such a case a complete pruning of LTP SELinux tests can be done and replaced with your tree contents,
2. If some of the LTP tests are not there, then i would like to retain them in LTP side-by-side your tree contents,

Quoted 7 lines

The new tree contains all of the selinux tests present in the LTP. You cannot however simply replace the ltp version of the tests with this tree, as this tree is a standalone testsuite and will not run within the ltp test harness. This testsuite is based on the original standalone selinux testsuite that was contributed by us to IBM to port to the LTP. We have internally maintained this testsuite in parallel to the ltp version as we have found it easier to set up, use, debug, and maintain.

Quoted 5 lines

Thanks Stephen. Since you have maintained the Original-One Internally and have simultaneously contributed changes to the LTP-version of SELinux, i am depending on you/Serge/SELinux-Mailing-List-members to keep sending patches to LTP to update the LTP-version at regular intervals (off-course at your convenience ;-))

Quoted 6 lines

I think that will depend on whether there is some benefit to maintaining the ltp version. We might derive some benefit if the ltp selinux testsuite were better integrated (e.g. tests run by default if SELinux is enabled on the host) and if the tests got some regular attention from the ltp maintainers. Otherwise, it may be better to just remove the tests from the ltp and point people to the standalone version.

I don't object to the tests being maintained in LTP. But so far, AFAICS, the cost of maintaining in LTP far outweighs the benefits.

So I'd like to know, does anyone (IBM?) *use* the version in LTP for automated testing?

If not, heck we could replace ltp/testcases/kernel/security/selinux/* with a script that git clones the testsuite and runs it.

Mind you it's not the selinux tests per se - it's the random bulk LTP updates which then break selinux tests, or worse, the autoconf cruft to try to detect whether the user wants selinux tests - which constitute the real maintenance cost.

If someone who uses the LTP selinux tests could step up and offer to periodically run the tests and work with (me and) the community to push fixes, I'll be happy to help out.

Quoted 58 lines · Serge E. Hallyn

Quoted 41 lines · Stephen Smalley

Quoted 34 lines · Subrata Modak

Quoted 25 lines · Stephen Smalley

Quoted 15 lines · Subrata Modak

Hi Serge,

Quoted 4 lines · Serge E. Hallyn

Just a little note to announce that the selinux testsuite is now up as a git tree at kernel.org.  You can fetch it using

    git clone git://git.kernel.org/pub/scm/tests/selinux-testsuite

Great. So, how far does it differ from that we have in LTP ?
http://ltp.git.sourceforge.net/git/gitweb.cgi?p=ltp/ltp-dev.git;a=tree;f=testcases/kernel/security/selinux-testsuite;h=066df4cdf2f8a80a0045e338b4bb2bf0f5d37091;hb=HEAD

As you, Stephen and others in SELinux <[email protected]> were the main contributors to the SELinux test(s) in LTP, i guess the next/future development for SELinux tests will be happening in the new tree.

Quoted 8 lines

So, i would rather keep pulling to LTP from your tree. Now, could you please let me know:

     1. Whether the new tree contains all scenarios which is present in LTP. In such a case a complete pruning of LTP SELinux tests can be done and replaced with your tree contents,
     2. If some of the LTP tests are not there, then i would like to retain them in LTP side-by-side your tree contents,

Quoted 7 lines

The new tree contains all of the selinux tests present in the LTP.  You cannot however simply replace the ltp version of the tests with this tree, as this tree is a standalone testsuite and will not run within the ltp test harness. This testsuite is based on the original standalone selinux testsuite that was contributed by us to IBM to port to the LTP. We have internally maintained this testsuite in parallel to the ltp version as we have found it easier to set up, use, debug, and maintain.

Quoted 5 lines

Thanks Stephen. Since you have maintained the Original-One Internally and have simultaneously contributed changes to the LTP-version of SELinux, i am depending on you/Serge/SELinux-Mailing-List-members to keep sending patches to LTP to update the LTP-version at regular intervals (off-course at your convenience ;-))

Quoted 6 lines

I think that will depend on whether there is some benefit to maintaining the ltp version.  We might derive some benefit if the ltp selinux testsuite were better integrated (e.g. tests run by default if SELinux is enabled on the host) and if the tests got some regular attention from the ltp maintainers.  Otherwise, it may be better to just remove the tests from the ltp and point people to the standalone version.

I don't object to the tests being maintained in LTP.  But so far, AFAICS, the cost of maintaining in LTP far outweighs the benefits.

So I'd like to know, does anyone (IBM?) *use* the version in LTP for automated testing?

If not, heck we could replace ltp/testcases/kernel/security/selinux/* with a script that git clones the testsuite and runs it.

Quoted 8 lines

Mind you it's not the selinux tests per se - it's the random bulk LTP updates which then break selinux tests, or worse, the autoconf cruft to try to detect whether the user wants selinux tests - which constitute the real maintenance cost.

If someone who uses the LTP selinux tests could step up and offer to periodically run the tests and work with (me and) the community to push fixes, I'll be happy to help out.

(cc'd Jeff Burke as I know he has run the ltp selinux tests in the past on RHEL and has contributed patches in the past)

Jeff - see above for context.  Do you still run the ltp selinux tests on a regular basis?  Are you interested in helping to maintain them and ideally to better integrate them into the LTP so that they will be run by default (if SELinux is enabled) and exercised by the regular ltp maintainers?  Or are you just as happy to clone the separate standalone selinux testsuite and run that as part of QA, separate from ltp testing?

Quoted 58 lines · Serge E. Hallyn

Quoted 41 lines · Stephen Smalley

Quoted 34 lines · Subrata Modak

Quoted 25 lines · Stephen Smalley

Quoted 15 lines · Subrata Modak

Hi Serge,

Quoted 4 lines · Serge E. Hallyn

Just a little note to announce that the selinux testsuite is now up as a git tree at kernel.org. You can fetch it using

git clone git://git.kernel.org/pub/scm/tests/selinux-testsuite

Great. So, how far does it differ from that we have in LTP ?
http://ltp.git.sourceforge.net/git/gitweb.cgi?p=ltp/ltp-dev.git;a=tree;f=testcases/kernel/security/selinux-testsuite;h=066df4cdf2f8a80a0045e338b4bb2bf0f5d37091;hb=HEAD

As you, Stephen and others in SELinux <[email protected]> were the main contributors to the SELinux test(s) in LTP, i guess the next/future development for SELinux tests will be happening in the new tree.

Quoted 8 lines

So, i would rather keep pulling to LTP from your tree. Now, could you please let me know:

1. Whether the new tree contains all scenarios which is present in LTP. In such a case a complete pruning of LTP SELinux tests can be done and replaced with your tree contents,
2. If some of the LTP tests are not there, then i would like to retain them in LTP side-by-side your tree contents,

Quoted 7 lines

The new tree contains all of the selinux tests present in the LTP. You cannot however simply replace the ltp version of the tests with this tree, as this tree is a standalone testsuite and will not run within the ltp test harness. This testsuite is based on the original standalone selinux testsuite that was contributed by us to IBM to port to the LTP. We have internally maintained this testsuite in parallel to the ltp version as we have found it easier to set up, use, debug, and maintain.

Quoted 5 lines

Thanks Stephen. Since you have maintained the Original-One Internally and have simultaneously contributed changes to the LTP-version of SELinux, i am depending on you/Serge/SELinux-Mailing-List-members to keep sending patches to LTP to update the LTP-version at regular intervals (off-course at your convenience ;-))

Quoted 6 lines

I think that will depend on whether there is some benefit to maintaining the ltp version. We might derive some benefit if the ltp selinux testsuite were better integrated (e.g. tests run by default if SELinux is enabled on the host) and if the tests got some regular attention from the ltp maintainers. Otherwise, it may be better to just remove the tests from the ltp and point people to the standalone version.

I don't object to the tests being maintained in LTP. But so far, AFAICS, the cost of maintaining in LTP far outweighs the benefits.

So I'd like to know, does anyone (IBM?) *use* the version in LTP for automated testing?

If not, heck we could replace ltp/testcases/kernel/security/selinux/* with a script that git clones the testsuite and runs it.

Quoted 8 lines

Mind you it's not the selinux tests per se - it's the random bulk LTP updates which then break selinux tests, or worse, the autoconf cruft to try to detect whether the user wants selinux tests - which constitute the real maintenance cost.

If someone who uses the LTP selinux tests could step up and offer to periodically run the tests and work with (me and) the community to push fixes, I'll be happy to help out.

(cc'd Jeff Burke as I know he has run the ltp selinux tests in the past on RHEL and has contributed patches in the past)

Jeff - see above for context. Do you still run the ltp selinux tests on a regular basis? Are you interested in helping to maintain them and ideally to better integrate them into the LTP so that they will be run by default (if SELinux is enabled) and exercised by the regular ltp

maintainers? Or are you just as happy to clone the separate standalone selinux testsuite and run that as part of QA, separate from ltp testing?

Quoted 41 lines · Stephen Smalley

Quoted 34 lines · Subrata Modak

Quoted 25 lines · Stephen Smalley

Quoted 15 lines · Subrata Modak

Hi Serge,

Quoted 4 lines · Serge E. Hallyn

Just a little note to announce that the selinux testsuite is now up as a git tree at kernel.org. You can fetch it using

git clone git://git.kernel.org/pub/scm/tests/selinux-testsuite

Great. So, how far does it differ from that we have in LTP ?
http://ltp.git.sourceforge.net/git/gitweb.cgi?p=ltp/ltp-dev.git;a=tree;f=testcases/kernel/security/selinux-testsuite;h=066df4cdf2f8a80a0045e338b4bb2bf0f5d37091;hb=HEAD

As you, Stephen and others in SELinux <[email protected]> were the main contributors to the SELinux test(s) in LTP, i guess the next/future development for SELinux tests will be happening in the new tree.

Quoted 8 lines

So, i would rather keep pulling to LTP from your tree. Now, could you please let me know:

1. Whether the new tree contains all scenarios which is present in LTP. In such a case a complete pruning of LTP SELinux tests can be done and replaced with your tree contents,
2. If some of the LTP tests are not there, then i would like to retain them in LTP side-by-side your tree contents,

Quoted 7 lines

The new tree contains all of the selinux tests present in the LTP. You cannot however simply replace the ltp version of the tests with this tree, as this tree is a standalone testsuite and will not run within the ltp test harness. This testsuite is based on the original standalone selinux testsuite that was contributed by us to IBM to port to the LTP. We have internally maintained this testsuite in parallel to the ltp version as we have found it easier to set up, use, debug, and maintain.

Quoted 5 lines

Thanks Stephen. Since you have maintained the Original-One Internally and have simultaneously contributed changes to the LTP-version of SELinux, i am depending on you/Serge/SELinux-Mailing-List-members to keep sending patches to LTP to update the LTP-version at regular intervals (off-course at your convenience ;-))

Quoted 6 lines

I think that will depend on whether there is some benefit to maintaining the ltp version. We might derive some benefit if the ltp selinux testsuite were better integrated (e.g. tests run by default if SELinux is enabled on the host) and if the tests got some regular attention from the ltp maintainers. Otherwise, it may be better to just remove the tests from the ltp and point people to the standalone version.

I don't object to the tests being maintained in LTP. But so far, AFAICS, the cost of maintaining in LTP far outweighs the benefits.

So I'd like to know, does anyone (IBM?) *use* the version in LTP for automated testing?

If not, heck we could replace ltp/testcases/kernel/security/selinux/* with a script that git clones the testsuite and runs it.

Mind you it's not the selinux tests per se - it's the random bulk LTP updates which then break selinux tests, or worse, the autoconf cruft to try to detect whether the user wants selinux tests - which constitute the real maintenance cost.

If someone who uses the LTP selinux tests could step up and offer to periodically run the tests and work with (me and) the community to push fixes, I'll be happy to help out.

-serge

--
This message was distributed to subscribers of the selinux mailing list. If you no longer wish to subscribe, send mail to [email protected] with the words "unsubscribe selinux" without quotes as the message.