futex syscall tests

LTP Mike Frysinger 3 participants 7 messages Jul 3, 2009 – Aug 13, 2009

Share & tools

Mike Frysinger

author

2009-07-03 20:02 UTC #0 lore raw

are there any futex syscall tests already in ltp that i cant seem to locate ? if not, i can adapt the few that already exist in the libfutex code ... -mike

------------------------------------------------------------------------------

_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Subrata Modak

2009-07-06 15:51 UTC #1 lore raw

Quoted 2 lines · Mike Frysinger

are there any futex syscall tests already in ltp that i cant seem to locate ? if not, i can adapt the few that already exist in the libfutex code ...

Mike, that should be really great to get to LTP.

Regards--
Subrata

Quoted 6 lines

-mike
------------------------------------------------------------------------------
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

------------------------------------------------------------------------------
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Subrata Modak

2009-07-30 12:44 UTC #2 lore raw

Quoted 3 lines · Subrata Modak

Quoted 2 lines · Mike Frysinger
are there any futex syscall tests already in ltp that i cant seem to locate ? if not, i can adapt the few that already exist in the libfutex code ...

Mike,

Did you post something on this, which i might have missed due to inactivity for last couple of weeks ?

Regards--
Subrata

Quoted 14 lines

Mike, that should be really great to get to LTP.
Regards--
Subrata
Quoted 6 lines
-mike
------------------------------------------------------------------------------
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Quoted 5 lines

------------------------------------------------------------------------------
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Darren Hart

2009-07-31 21:30 UTC #3 lore raw

In reply to Mike Frysinger

Quoted 3 lines · Mike Frysinger

are there any futex syscall tests already in ltp that i cant seem to locate ? if not, i can adapt the few that already exist in the libfutex code ... -mike

Mike,

I recently done a lot of cleanup and a couple feature additions to the kernel futex code. In doing so I was noticing the lack of a good futex system call test suite. I had started a writeup of what such a test suite should accomplish. I hadn't decided if I wanted to push it to LTP eventually or not. As it sounds like you may have similar interests in this test, please find my very early, very incomplete, initial thoughts on such a testsuite.

I'd welcome your thoughts on approach, direction, etc.

-------------------------------------------------------------------------------

The following is a start at what I hope to be a design document for a new exhaustive futex functional/performance/stress benchmark. Before I flesh it out any further, I'd really appreciate some feedback on what I have so far.

Given the number of permutations each test seems to require, I think some kind of "test generator" is needed. We will have to describe the permutations and pass them to the test generator so it can create the various versions of the tests.

HAVOC FUTEX
===========

Havoc Futex is intended to thoroughly test the Linux kernel futex system call API. To the extent possible, each test is implemented using raw system calls as well as the glibc pthread library.

Futex Op Codes
--------------
FUTEX_WAIT
FUTEX_WAKE
FUTEX_FD
FUTEX_REQUEUE
FUTEX_CMP_REQUEUE
FUTEX_WAKE_OP
FUTEX_LOCK_PI
FUTEX_UNLOCK_PI
FUTEX_TRYLOCK_PI
FUTEX_WAIT_BITSET
FUTEX_WAKE_BITSET
FUTEX_WAIT_REQUEUE_PI
FUTEX_CMP_REQUEUE_PI

Syscalls to Test
----------------
futex_wake
futex_wake_op
futex_requeue
futex_wait
futex_lock_pi
futex_unlock_pi
futex_wait_requeue_pi

Functional Tests
----------------
requeue_pi/*
Exercise the FUTEX_WAIT_REQUEUE_PI and FUTEX_CMP_REQUEUE_PI op codes, under every possible combination of the following scenarios:

o shared and private futexes
o CLOCK_MONOTONIC and CLOCK_REALTIME timeouts (and none)
o Signal handling prior to and post requeue
o correct and incorrect settings for val
o target futex owned by waker, owned by third party, unowned
o OWNERDIED reclaim of mutex
o ensure priority ordered wakeup of waiters

Error and Misuse Cases
----------------------
o mixed shared and private futexes (should fail)
o pi source futex
o non-pi target futex
o unmapped shared futex fault handling
o bogus uaddrs
o invalid nr_wake and nr_requeue values
o mismatched wait_requeue and futex_requeue target futexes
o incorrect pairing of futex_wait_requeue_pi with futex_wake

Syscalls Exercised
------------------
futex_wait_requeue_pi
futex_requeue
futex_lock_pi
futex_unlock_pi

pi_lock/*
Exercise the FUTEX_LOCK_PI and FUTEX_UNLOCK_PI op codes, under every possible combination of the following scenarious:

o shared and private futexes
o CLOCK_MONOTONIC and CLOCK_REALTIME timeouts (and none)
o Signal handling
o correct and incorrect settings for val
o bogus uaddrs
o contended and uncontended cases
o OWNERDIED reclaim of mutex

Error and Misuse Cases
----------------------
o pi_unlock of a non-pi-locked futex
o pi_lock of an owned non-pi futex
o unmapped shared futex fault handling
o mismatched futex_lock_pi and futex_wake(_op)? calls
o mismatched futex_wait and futex_unlock_pi calls

Syscalls Exercised
------------------
futex_lock_pi
futex_unlock_pi
futex_wait
futex_wake
futex_wake_op

requeue/*
Exercise the FUTEX_WAIT and the FUTEX_CMP_REQUEUE op codes. Perform basic testing for FUTEX_REQUEUE, purposefully avoiding its known flaws.

Error and Misuse Cases
----------------------

Syscalls Exercised
------------------
futex_wait
futex_requeue

wait/*

	Exercise the FUTEX_WAIT and FUTEX_WAKE op codes.

	Error and Misuse Cases
	----------------------

	Syscalls Exercised
	------------------
	futex_wait
	futex_wake
	futex_wake_op

Performance Tests
-----------------
o attempt to expose lock contention issues, such as those exposed by calling futex_wait on an unowned futex
o rapid lock and unlock of an uncontended futex
o rapid lock and unlock of a heavily conteded futex
o attempt to expose bottlenecks imposed by the shared hash-bucket implementation
o attempt to expose real-time scheduling overhead

Stress Tests
------------
o thousands of threads/processes contending on a single futex
o thousands of threads/processes on thousands of futexes

Other Thoughts
--------------
kernel-side futex fault injection
There are a lot of places in futex.c that have to handle faults. I think some kind of a fault injection system is needed. This could be enabled via a sysctl or perhaps just configured in to a debug kernel. Running this test suite in a loop would allow us to achieve some statistical confidence in these numerous fault paths.

FUTEX_REQUEUE
This op code is deprecated in favor of FUTEX_CMP_REQUEUE. Do to the unreliable nature of the op code, only very limited testing can be performed.

Signature · 13 lines

Darren Hart
IBM Linux Technology Center
Real-Time Linux Team

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Subrata Modak

2009-08-03 11:21 UTC #4 lore raw

Hi Darren,

Quoted 11 lines · Darren Hart

Quoted 3 lines · Mike Frysinger
are there any futex syscall tests already in ltp that i cant seem to locate ? if not, i can adapt the few that already exist in the libfutex code ... -mike
Mike,
I recently done a lot of cleanup and a couple feature additions to the kernel futex code. In doing so I was noticing the lack of a good futex system call test suite. I had started a writeup of what such a test suite should accomplish. I hadn't decided if I wanted to push it to LTP

I would request you to eventually push it to LTP. I think Mike might be interested in reviewing the initial test code and then suggest his additions/deletions. That way it would work fine, until Mike has already written something for this.

And thanks for sharing the below design information.

Regards--
Subrata

Quoted 161 lines

eventually or not. As it sounds like you may have similar interests in this test, please find my very early, very incomplete, initial thoughts on such a testsuite.
I'd welcome your thoughts on approach, direction, etc.
-------------------------------------------------------------------------------
The following is a start at what I hope to be a design document for a new exhaustive futex functional/performance/stress benchmark. Before I flesh it out any further, I'd really appreciate some feedback on what I have so far.
Given the number of permutations each test seems to require, I think some kind of "test generator" is needed. We will have to describe the permutations and pass them to the test generator so it can create the various versions of the tests.
HAVOC FUTEX
===========
Havoc Futex is intended to thoroughly test the Linux kernel futex system call API. To the extent possible, each test is implemented using raw system calls as well as the glibc pthread library.
Futex Op Codes
--------------
FUTEX_WAIT
FUTEX_WAKE
FUTEX_FD
FUTEX_REQUEUE
FUTEX_CMP_REQUEUE
FUTEX_WAKE_OP
FUTEX_LOCK_PI
FUTEX_UNLOCK_PI
FUTEX_TRYLOCK_PI
FUTEX_WAIT_BITSET
FUTEX_WAKE_BITSET
FUTEX_WAIT_REQUEUE_PI
FUTEX_CMP_REQUEUE_PI
Syscalls to Test
----------------
futex_wake
futex_wake_op
futex_requeue
futex_wait
futex_lock_pi
futex_unlock_pi
futex_wait_requeue_pi
Functional Tests
----------------
requeue_pi/*
Exercise the FUTEX_WAIT_REQUEUE_PI and FUTEX_CMP_REQUEUE_PI op codes, under every possible combination of the following scenarios:
o shared and private futexes
o CLOCK_MONOTONIC and CLOCK_REALTIME timeouts (and none)
o Signal handling prior to and post requeue
o correct and incorrect settings for val
o target futex owned by waker, owned by third party, unowned
o OWNERDIED reclaim of mutex
o ensure priority ordered wakeup of waiters
Error and Misuse Cases
----------------------
o mixed shared and private futexes (should fail)
o pi source futex
o non-pi target futex
o unmapped shared futex fault handling
o bogus uaddrs
o invalid nr_wake and nr_requeue values
o mismatched wait_requeue and futex_requeue target futexes
o incorrect pairing of futex_wait_requeue_pi with futex_wake
Syscalls Exercised
------------------
futex_wait_requeue_pi
futex_requeue
futex_lock_pi
futex_unlock_pi
pi_lock/*
Exercise the FUTEX_LOCK_PI and FUTEX_UNLOCK_PI op codes, under every possible combination of the following scenarious:
o shared and private futexes
o CLOCK_MONOTONIC and CLOCK_REALTIME timeouts (and none)
o Signal handling
o correct and incorrect settings for val
o bogus uaddrs
o contended and uncontended cases
o OWNERDIED reclaim of mutex
Error and Misuse Cases
----------------------
o pi_unlock of a non-pi-locked futex
o pi_lock of an owned non-pi futex
o unmapped shared futex fault handling
o mismatched futex_lock_pi and futex_wake(_op)? calls
o mismatched futex_wait and futex_unlock_pi calls
Syscalls Exercised
------------------
futex_lock_pi
futex_unlock_pi
futex_wait
futex_wake
futex_wake_op
requeue/*
Exercise the FUTEX_WAIT and the FUTEX_CMP_REQUEUE op codes. Perform basic testing for FUTEX_REQUEUE, purposefully avoiding its known flaws.
Error and Misuse Cases
----------------------
Syscalls Exercised
------------------
futex_wait
futex_requeue
wait/*
	Exercise the FUTEX_WAIT and FUTEX_WAKE op codes.

	Error and Misuse Cases
	----------------------

	Syscalls Exercised
	------------------
	futex_wait
	futex_wake
	futex_wake_op
Performance Tests
-----------------
o attempt to expose lock contention issues, such as those exposed by calling futex_wait on an unowned futex
o rapid lock and unlock of an uncontended futex
o rapid lock and unlock of a heavily conteded futex
o attempt to expose bottlenecks imposed by the shared hash-bucket implementation
o attempt to expose real-time scheduling overhead
Stress Tests
------------
o thousands of threads/processes contending on a single futex
o thousands of threads/processes on thousands of futexes
Other Thoughts
--------------
kernel-side futex fault injection
There are a lot of places in futex.c that have to handle faults. I think some kind of a fault injection system is needed. This could be enabled via a sysctl or perhaps just configured in to a debug kernel. Running this test suite in a loop would allow us to achieve some statistical confidence in these numerous fault paths.

Quoted 5 lines

FUTEX_REQUEUE
This op code is deprecated in favor of FUTEX_CMP_REQUEUE. Do to the unreliable nature of the op code, only very limited testing can be performed.

Mike Frysinger

author

2009-08-13 09:09 UTC #5 lore raw

In reply to Darren Hart

Quoted 5 lines · On Friday 31 July 2009 17:29:55 Darren Hart wrote:

Quoted 3 lines · Mike Frysinger
are there any futex syscall tests already in ltp that i cant seem to locate ? if not, i can adapt the few that already exist in the libfutex code ...

Quoted 7 lines

I recently done a lot of cleanup and a couple feature additions to the kernel futex code. In doing so I was noticing the lack of a good futex system call test suite. I had started a writeup of what such a test suite should accomplish. I hadn't decided if I wanted to push it to LTP eventually or not. As it sounds like you may have similar interests in this test, please find my very early, very incomplete, initial thoughts on such a testsuite.

i noticed recently the checkpoint/resume guys seem to have already started some. google for "c/r futex".
-mike

_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Darren Hart

2009-08-13 16:08 UTC #6 lore raw

Quoted 13 lines · Mike Frysinger

Quoted 11 lines · On Friday 31 July 2009 17:29:55 Darren Hart wrote:
Quoted 3 lines · Mike Frysinger
are there any futex syscall tests already in ltp that i cant seem to locate ? if not, i can adapt the few that already exist in the libfutex code ...
I recently done a lot of cleanup and a couple feature additions to the kernel futex code. In doing so I was noticing the lack of a good futex system call test suite. I had started a writeup of what such a test suite should accomplish. I hadn't decided if I wanted to push it to LTP eventually or not. As it sounds like you may have similar interests in this test, please find my very early, very incomplete, initial thoughts on such a testsuite.

Quoted 2 lines

i noticed recently the checkpoint/resume guys seem to have already started some. google for "c/r futex".

:-) One of those guys is on CC actually and he is the one who suggested to me that I send my initial designs to the list. The goals are slightly different I think, but there is some overlap. We should definitely keep eachother in the loop. I admit to not having much time to dedicate to this, but I do feel an exhaustive futex syscall testsuite is very much needed, especially as the scope of futexes has been expanding.

Signature · 13 lines

Darren Hart
IBM Linux Technology Center
Real-Time Linux Team

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list