multipath-tools: generic async threads for TUR checker
v3 cover multipath-tools: generic async threads for TUR checker
Motivated by the recent submission of an ALUA checker for multipath-tools by Brian Bunker, I am proposing a generic framework for asynchronous checker threads in multipath-tools.
The first patch is a small fix I came up with while testing this. 2/5 is the actual implementation, 3/5 test code, and 4/4 modifies the TUR checker to the new library code. 5/5 is new, it simplifies testing the timeout handling code.
This makes the logic of the TUR checker easier to understand, and should make it easier for Brian to implement the ALUA checker based on the same framework.
Further improvements on top of this are possible. The new TUR code, except for the tur_check() function itself, is pretty generic and would allow abstracting an "async checker" model with just a few changes. This would make it possible to switch also the legacy checkers to an asynchronous mode of operation.
Comments and reviews welcome.
Martin
Changes v2 -> v3 (based on the review by Ben Marzinski):
- 02/05: fix handling of cancellation in idle state: don't attempt to increase refcount or free context in runner_thread(). Set refcount to 2 to begin with.
- 03/05: treat "-t 0" option as minimal, not infinite, timeout, to enable testing early cancellation.
- add compile time option RUNNER_START_DELAY_US=<N> for testing races between thread cancellation and startup
Changes v1 -> v2 (based on the review by Ben Marzinski):
- The design of the runner code has been changed such that no caller-side references are dropped automatically any more. The caller needs
to call release_runner() explicitly to drop its reference, independent of the runner status. This simplifies the code and allows implementing the MAX_TIMEOUTS logic in tur.c as it used to be. The overall API is not more complex, as release_runner() replaces cancel_runner().
- An additional state RUNNER_DEAD is introduced, denoting a state in which the callback result is unavailable, but the thread has finished.
- The naming of variables and struct members has been cleaned up.
- 02/05: specified alignment for struct runner_context.data
- 02/05: removed most assert() statements
- 02/05: removed superfluous cmm_smp_wmb()
- 02/05: perform all state modifications in the runner's cleanup function
- 03/05: fixed signal handler name
- 03/05: reduced the run time of the runner test script
- 04/05: revert back to the original MAX_TIMEOUTS logic, keeping a ref the "last" runner context until the thread eventually terminates
- 04/05: consistent handling of state and checker message
- 05/05: new
Martin Wilck (5):
multipathd: get_new_state: map PATH_TIMEOUT to PATH_DOWN
libmpathutil: add generic implementation for checker thread runners multipath-tools tests: add test program for thread runners
libmultipath: TUR checker: use runner threads
libmultipath: tur checker: improve tur_deep_sleep() test
libmpathutil/Makefile | 2 +- libmpathutil/libmpathutil.version | 7 +- libmpathutil/runner.c | 229 ++++++++++++ libmpathutil/runner.h | 102 ++++++ libmultipath/checkers/tur.c | 369 ++++++------------- multipathd/main.c | 4 +- tests/Makefile | 15 +- tests/runner-test.sh | 57 +++ tests/runner-test.supp | 15 + tests/runner.c | 564 ++++++++++++++++++++++++++++++ 10 files changed, 1103 insertions(+), 261 deletions(-) create mode 100644 libmpathutil/runner.c create mode 100644 libmpathutil/runner.h create mode 100755 tests/runner-test.sh create mode 100644 tests/runner-test.supp create mode 100644 tests/runner.c
Signature · 1 line
2.53.0
PATCH v3 1/5 multipathd: get_new_state: map PATH_TIMEOUT to PATH_DOWN
We map PATH_TIMEOUT to PATH_DOWN in pathinfo(), but not in get_new_state(). Do it there, too, to treat the states consistently.
This avoids logging "checker timed out" twice in update_path_state(), even if log_checker_err is set to "once".
--- multipathd/main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
multipathd/main.c +3 -1
@@ -2503,8 +2503,10 @@ get_new_state(struct path *pp)* Wait for uevent for removed paths; * some LLDDs like zfcp keep paths unavailable * without sending uevents. +* Also, map PATH_TIMEOUT to PATH_DOWN here, like we do in +* pathinfo(). */ -if (newstate ==PATH_REMOVED ) +if (newstate ==PATH_REMOVED || newstate ==PATH_TIMEOUT ) newstate =PATH_DOWN ;/*
Signature · 1 line
2.53.0
PATCH v3 2/5 libmpathutil: add generic implementation for checker thread runners
In reply to Martin WilckThis code adds a generic, abstract implementation of the kind of threads we need for checkers: detached threads that may time out. The design is such that these threads never access any memory of the calling program, simplifying the management of lifetimes of objects.
See the documentation of the API in "runner.h".
--- libmpathutil/Makefile | 2 +- libmpathutil/libmpathutil.version | 7 +- libmpathutil/runner.c | 229 ++++++++++++++++++++++++++++++ libmpathutil/runner.h | 102 +++++++++++++ 4 files changed, 338 insertions(+), 2 deletions(-) create mode 100644 libmpathutil/runner.c create mode 100644 libmpathutil/runner.h
libmpathutil/Makefile +1 -1
@@ -13,7 +13,7 @@ LIBDEPS += -lpthread -ldl -ludev -L$(mpathcmddir) -lmpathcmd $(SYSTEMD_LIBDEPS) # other object files OBJS := mt-libudev.o parser.o vector.o util.o debug.o time-util.o \ - uxsock.o log_pthread.o log.o strbuf.o globals.o msort.o + uxsock.o log_pthread.o log.o strbuf.o globals.o msort.o runner.o all: $(DEVLIB)
libmpathutil/libmpathutil.version +6 -1
@@ -43,7 +43,7 @@ LIBMPATHCOMMON_1.0.0 { put_multipath_config; }; -LIBMPATHUTIL_5.0 { +LIBMPATHUTIL_6.0 { global: alloc_bitfield; alloc_strvec;
@@ -62,6 +62,7 @@ global: cleanup_vector; cleanup_vector_free; convert_dev; + check_runner; dlog; filepresent; fill_strbuf;
@@ -72,6 +73,8 @@ global: free_strvec; get_linux_version_code; get_monotonic_time; + get_persistent_runner; + get_runner; get_strbuf_buf__; get_next_string; get_strbuf_len;
@@ -161,7 +164,9 @@ global: process_file; pthread_cond_init_mono; recv_packet; + release_runner; reset_strbuf; + runner_state_name; safe_write; send_packet; set_max_fds;
libmpathutil/runner.c new file +229 -0
229 lines not on this page · raw message
libmpathutil/runner.h new file +102 -0
102 lines not on this page · raw message
Signature · 1 line
2.53.0
Re: [PATCH v3 2/5] libmpathutil: add generic implementation for checker thread runners
Quoted 8 lines · Martin Wilck
This code adds a generic, abstract implementation of the kind of threads we need for checkers: detached threads that may time out. The design is such that these threads never access any memory of the calling program, simplifying the management of lifetimes of objects.
See the documentation of the API in "
runner.h".Signed-off-by Martin Wilck <[email protected]>
PATCH v3 3/5 multipath-tools tests: add test program for thread runners
In reply to Martin WilckAdd a test program for the "runner" thread implementation from the previous commit. The test program runs simulated "hanging" threads that may time out, and optionally kills all threads at an arbitrary point in time. See the comments at the top of the file for details.
Also add a test driver script (runner-test.sh) with a few reasonable combinations of command line arguments.
The test program has been used to test the "runner" implementation extensively on different architectures (x86_64, aarch64, ppc64le, s390x), using both valgrind and the gcc address sanitizer (libasan) for detection of memory leaks and use-after-free errors.
For valgrind, a suppression file needs to be added, as valgrind doesn't seem to capture the deallocation of thread local storage for detached threads in the test case where the test program is killed. The suppression affects only memory allocated by glibc. This leak has not been seen with libasan, only with valgrind.
--- tests/Makefile | 15 +- tests/runner-test.sh | 57 +++++ tests/runner-test.supp | 15 ++ tests/runner.c | 564 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 647 insertions(+), 4 deletions(-) create mode 100755 tests/runner-test.sh create mode 100644 tests/runner-test.supp create mode 100644 tests/runner.c
tests/Makefile +11 -4
@@ -9,12 +9,12 @@ CFLAGS += $(BIN_CFLAGS) -Wno-unused-parameter -Wno-unused-function $(W_MISSING_I LIBDEPS += -L. -L $(mpathutildir) -L$(mpathcmddir) -lmultipath -lmpathutil -lmpathcmd -lcmocka TESTS := uevent parser util dmevents hwtable blacklist unaligned vpd pgpolicy \ - alias directio valid devt mpathvalid strbuf sysfs features cli mapinfo + alias directio valid devt mpathvalid strbuf sysfs features cli mapinfo runner HELPERS := test-lib.o test-log.o .PRECIOUS: $(TESTS:%=%-test) -all: $(TESTS:%=%.out) +all: $(TESTS:%=%.out) runner-test.out progs: $(TESTS:%=%-test) lib/libchecktur.so valgrind: $(TESTS:%=%.vgr)
@@ -65,6 +65,7 @@ features-test_LIBDEPS := -ludev -lpthread features-test_OBJDEPS := $(mpathutildir)/mt-libudev.o cli-test_OBJDEPS := $(daemondir)/cli.o mapinfo-test_LIBDEPS = -lpthread -ldevmapper +runner-test_LIBDEPS = -lpthread %.o: %.c @echo building $@ because of $?
@@ -81,12 +82,18 @@ lib/libchecktur.so: %.vgr: %-test lib/libchecktur.so @echo == running valgrind for $< == @LD_LIBRARY_PATH=.:$(mpathutildir):$(mpathcmddir) \ - valgrind --leak-check=full --error-exitcode=128 ./$< >$@ 2>&1 + valgrind --leak-check=full --error-exitcode=128 --suppressions=./runner-test.supp ./$< >$@ 2>&1 OBJS = $(TESTS:%=%.o) $(HELPERS) +runner-test.out: runner-test + $(Q)./runner-test.sh >$@ 2>&1 + +runner-test.vgr: runner-test + $(Q)VALGRIND=1 ./runner-test.sh >$@ 2>&1 + test_clean: - $(Q)$(RM) $(TESTS:%=%.out) $(TESTS:%=%.vgr) *.so* + $(Q)$(RM) $(TESTS:%=%.out) $(TESTS:%=%.vgr) *.so* runner-test.out runner-test.vgr valgrind_clean: $(Q)$(RM) $(TESTS:%=%.vgr)
tests/runner-test.sh new file +57 -0
57 lines not on this page · raw message
tests/runner-test.supp new file +15 -0
@@ -0,0 +1,15 @@ +{ + glibc TLS for detached threads if program is terminated + Memcheck:Leak + match-leak-kinds: possible + fun:calloc + ... + fun:allocate_dtv + fun:_dl_allocate_tls + fun:allocate_stack + ... + fun:get_runner + fun:start_runner + ... + fun:main +}
tests/runner.c new file +564 -0
564 lines not on this page · raw message
Signature · 1 line
2.53.0
Re: [PATCH v3 3/5] multipath-tools tests: add test program for thread runners
Quoted 13 lines · Martin Wilck
Add a test program for the "runner" thread implementation from the previous commit. The test program runs simulated "hanging" threads that may time out, and optionally kills all threads at an arbitrary point in time. See the comments at the top of the file for details.
Also add a test driver script (
runner-test.sh) with a few reasonable combinations of command line arguments.The test program has been used to test the "runner" implementation extensively on different architectures (
x86_64, aarch64, ppc64le, s390x), using both valgrind and the gcc address sanitizer (libasan) for detection of memory leaks and use-after-free errors.
Quoted 7 lines
For valgrind, a suppression file needs to be added, as valgrind doesn't seem to capture the deallocation of thread local storage for detached threads in the test case where the test program is killed. The suppression affects only memory allocated by glibc. This leak has not been seen with libasan, only with valgrind.
Signed-off-by Martin Wilck <[email protected]>
PATCH v3 4/5 libmultipath: TUR checker: use runner threads
In reply to Martin WilckUse the generic runners to simplify the TUR checker code.
--- libmultipath/checkers/tur.c | 363 +++++++++++------------------------- 1 file changed, 112 insertions(+), 251 deletions(-)
libmultipath/checkers/tur.c +112 -251
@@ -3,7 +3,6 @@* * Copyright (c) 2004 Christophe Varoqui */ -#include <stdio.h> #include <stdlib.h> #include <string.h> #include <sys/types.h>
@@ -14,16 +13,11 @@#include <sys/sysmacros.h> #include <errno.h> #include <sys/time.h> -#include <pthread.h> -#include <urcu.h> -#include <urcu/uatomic.h> #include "checkers.h" -#include "debug.h" #include "sg_include.h" -#include "util.h" -#include "time-util.h" +#include "runner.h" #define TUR_CMD_LEN 6#define HEAVY_CHECK_COUNT 10
@@ -45,70 +39,47 @@ const char *libcheck_msgtable[] = { NULL, }; -struct tur_checker_context { -dev_t devt; -int state; -int running;/* uatomic access only */ +struct tur_data {int fd; +dev_t devt;unsigned int timeout; -time_t time; -pthread_t thread; -pthread_mutex_t lock; -pthread_cond_t active; -int holders;/* uatomic access only */ -int msgid; -struct checker_context ctx; -unsigned int nr_timeouts; -bool checked_state; +int state; +short msgid; }; -int libcheck_init (struct checker * c) +struct tur_checker_context { +struct checker_context chkr; +int last_runner_state; +unsigned int nr_timeouts; +struct runner_context *rtx; +struct tur_data tdata; +}; + +int libcheck_init (struct checker *c) { -struct tur_checker_context *ct; +struct tur_checker_context *tcc;struct stat sb; - ct =malloc (sizeof (struct tur_checker_context )); -if (!ct) -return 1 ; -memset (ct,0 ,sizeof (struct tur_checker_context )); - - ct->state =PATH_UNCHECKED ; - ct->fd =-1 ; -uatomic_set (&ct->holders,1 ); -pthread_cond_init_mono (&ct->active); -pthread_mutex_init (&ct->lock,NULL ); + tcc =calloc (1 ,sizeof (*tcc)); + tcc->tdata.state =PATH_UNCHECKED ; + tcc->tdata.fd =-1 ;if (fstat (c->fd, &sb) ==0 ) - ct->devt = sb.st_rdev; - ct->ctx.cls = c->cls; - c->context = ct; - + tcc->tdata.devt = sb.st_rdev; + tcc->chkr.cls = c->cls; + c->context = tcc;return 0 ; } -static void cleanup_context (struct tur_checker_context *ct) -{ -pthread_mutex_destroy (&ct->lock); -pthread_cond_destroy (&ct->active); -free (ct); -} -void libcheck_free (struct checker * c) { -if (c->context) { -struct tur_checker_context *ct = c->context; -int holders; -int running; +struct tur_checker_context *tcc = c->context; - running =uatomic_xchg (&ct->running,0 ); -if (running) -pthread_cancel (ct->thread); - ct->thread =0 ; - holders =uatomic_sub_return (&ct->holders,1 ); -if (!holders) -cleanup_context (ct); - c->context =NULL ; - } -return ; +if (!tcc) +return ; + c->context =NULL ; +if (tcc->rtx) +release_runner (tcc->rtx); +free (tcc); } static int
@@ -216,19 +187,6 @@ retry:return PATH_UP ; } -#define tur_thread_cleanup_push (ct) pthread_cleanup_push(cleanup_func, ct) -#define tur_thread_cleanup_pop (ct) pthread_cleanup_pop(1) - -static void cleanup_func (void *data) -{ -int holders; -struct tur_checker_context *ct = data; - - holders =uatomic_sub_return (&ct->holders,1 ); -if (!holders) -cleanup_context (ct); -} -/* * Test code for "zombie tur thread" handling. * Compile e.g. with CFLAGS=-DTUR_TEST_MAJOR=8
@@ -249,13 +207,13 @@ static void cleanup_func(void *data)#define TUR_SLEEP_SECS 60#endif -static void tur_deep_sleep (const struct tur_checker_context *ct) +static void tur_deep_sleep (const struct tur_data *tdata) {static int sleep_cnt;const struct timespec ts = { .tv_sec =TUR_SLEEP_SECS , .tv_nsec =0 };int oldstate; -if (ct->devt !=makedev (TUR_TEST_MAJOR ,TUR_TEST_MINOR ) || +if (tdata->devt !=makedev (TUR_TEST_MAJOR ,TUR_TEST_MINOR ) || ++sleep_cnt %TUR_SLEEP_INTERVAL !=0 )return ;
@@ -273,110 +231,92 @@ static void tur_deep_sleep(const struct tur_checker_context *ct)#define tur_deep_sleep (x) do {} while (0)#endif /* TUR_TEST_MAJOR */ -void *libcheck_thread (struct checker_context *ctx) +void runner_callback (void *arg) { -struct tur_checker_context *ct = - container_of(ctx, struct tur_checker_context, ctx); -int state, running; -short msgid; +struct tur_data *tdata = arg; +int state; -/* This thread can be canceled, so setup clean up */ -tur_thread_cleanup_push (ct); +condlog (4 ,"%d:%d : tur checker starting up" ,major (tdata->devt), +minor (tdata->devt)); -condlog (4 ,"%d:%d : tur checker starting up" ,major (ct->devt), -minor (ct->devt)); - -tur_deep_sleep (ct); - state =tur_check (ct->fd, ct->timeout, &msgid); +tur_deep_sleep (tdata); + state =tur_check (tdata->fd, tdata->timeout, &tdata->msgid); + tdata->state = state;pthread_testcancel (); - -/* TUR checker done */ -pthread_mutex_lock (&ct->lock); - ct->state = state; - ct->msgid = msgid; -pthread_cond_signal (&ct->active); -pthread_mutex_unlock (&ct->lock); - -condlog (4 ,"%d:%d : tur checker finished, state %s" ,major (ct->devt), -minor (ct->devt),checker_state_name (state)); - - running =uatomic_xchg (&ct->running,0 ); -if (!running) -pause (); - -tur_thread_cleanup_pop (ct); - -return ((void *)0 ); +condlog (4 ,"%d:%d : tur checker finished, state %s" ,major (tdata->devt), +minor (tdata->devt),checker_state_name (state)); } -static void tur_set_async_timeout (struct checker *c) +static int check_runner_state (struct tur_checker_context *tcc) { -struct tur_checker_context *ct = c->context; -struct timespec now; +struct runner_context *rtx = tcc->rtx; +int rc; -get_monotonic_time (&now); - ct->time = now.tv_sec + c->timeout; -} - -static int tur_check_async_timeout (struct checker *c) -{ -struct tur_checker_context *ct = c->context; -struct timespec now; - -get_monotonic_time (&now); -return (now.tv_sec > ct->time); -} - -int check_pending (struct checker *c) -{ -struct tur_checker_context *ct = c->context; -int tur_status =PATH_PENDING ; - -pthread_mutex_lock (&ct->lock); - -if (ct->state !=PATH_PENDING || ct->msgid !=MSG_TUR_RUNNING ) - { - tur_status = ct->state; - c->msgid = ct->msgid; - } -pthread_mutex_unlock (&ct->lock); -if (tur_status ==PATH_PENDING && c->msgid ==MSG_TUR_RUNNING ) { + rc =check_runner (rtx, &tcc->tdata,sizeof (tcc->tdata)); +switch (rc) { +case RUNNER_DEAD : + tcc->tdata.state =PATH_TIMEOUT ; + tcc->tdata.msgid =MSG_TUR_TIMEOUT ; +/* fallthrough */ +case RUNNER_DONE : +release_runner (tcc->rtx); + tcc->rtx =NULL ; + tcc->last_runner_state = rc; + tcc->nr_timeouts =0 ; +condlog (rc ==RUNNER_DONE ?4 :3 , +"%d:%d : tur checker finished, state %s, runner state %s" , +major (tcc->tdata.devt),minor (tcc->tdata.devt), +checker_state_name (tcc->tdata.state), +runner_state_name (rc)); +break ; +case RUNNER_CANCELLED : + tcc->last_runner_state = rc; + tcc->tdata.state =PATH_TIMEOUT ; + tcc->tdata.msgid =MSG_TUR_TIMEOUT ; +if (tcc->nr_timeouts <MAX_NR_TIMEOUTS ) { +condlog (3 ,"%d:%d : tur checker timed out, releasing it" , +major (tcc->tdata.devt),minor (tcc->tdata.devt)); + tcc->nr_timeouts++; +release_runner (tcc->rtx); + tcc->rtx =NULL ; + }else if (tcc->nr_timeouts ==MAX_NR_TIMEOUTS ) { + tcc->nr_timeouts++; +condlog (3 ,"%d:%d : tur checker timed out, waiting for it" , +major (tcc->tdata.devt),minor (tcc->tdata.devt)); + } +break ; +default :condlog (4 ,"%d:%d : tur checker still running" , -major (ct->devt),minor (ct->devt)); - }else { -int running =uatomic_xchg (&ct->running,0 ); -if (running) -pthread_cancel (ct->thread); - ct->thread =0 ; +major (tcc->tdata.devt),minor (tcc->tdata.devt)); + tcc->tdata.msgid =MSG_TUR_RUNNING ; +break ; } - - ct->checked_state = true; -return tur_status; +return rc; }bool libcheck_need_wait (struct checker *c) {struct tur_checker_context *ct = c->context; -return (ct && ct->thread &&uatomic_read (&ct->running) !=0 && - !ct->checked_state); + +return ct && ct->rtx; }int libcheck_pending (struct checker *c) {struct tur_checker_context *ct = c->context; -/* The if path checker isn't running, just return the exiting value. */ -if (!ct || !ct->thread) +if (!ct || !ct->rtx)return c->path_state; -return check_pending (c); +/* This may nullify ct->rtx */ +check_runner_state (ct); + c->msgid = ct->tdata.msgid; +return ct->tdata.state; }int libcheck_check (struct checker * c) {struct tur_checker_context *ct = c->context; -pthread_attr_t attr; -int tur_status, r;if (!ct)return PATH_UNCHECKED ;
@@ -384,109 +324,30 @@ int libcheck_check(struct checker * c)if (checker_is_sync (c))return tur_check (c->fd, c->timeout, &c->msgid); -/* -* Async mode -*/ -if (ct->thread) { - ct->checked_state = true; -if (tur_check_async_timeout (c)) { -int running =uatomic_xchg (&ct->running,0 ); -if (running) { -pthread_cancel (ct->thread); -condlog (3 ,"%d:%d : tur checker timeout" , -major (ct->devt),minor (ct->devt)); - c->msgid =MSG_TUR_TIMEOUT ; - tur_status =PATH_TIMEOUT ; - }else { -pthread_mutex_lock (&ct->lock); - tur_status = ct->state; - c->msgid = ct->msgid; -pthread_mutex_unlock (&ct->lock); - } - ct->thread =0 ; - }else if (uatomic_read (&ct->running) !=0 ) { -condlog (3 ,"%d:%d : tur checker not finished" , -major (ct->devt),minor (ct->devt)); - tur_status =PATH_PENDING ; - c->msgid =MSG_TUR_RUNNING ; - }else { -/* TUR checker done */ - ct->thread =0 ; -pthread_mutex_lock (&ct->lock); - tur_status = ct->state; - c->msgid = ct->msgid; -pthread_mutex_unlock (&ct->lock); - } - }else { -if (uatomic_read (&ct->holders) >1 ) { -/* The thread has been cancelled but hasn't quit. */ -if (ct->nr_timeouts ==MAX_NR_TIMEOUTS ) { -condlog (2 ,"%d:%d : waiting for stalled tur thread to finish" , -major (ct->devt),minor (ct->devt)); - ct->nr_timeouts++; - } -/* -* Don't start new threads until the last once has -* finished. -*/ -if (ct->nr_timeouts >MAX_NR_TIMEOUTS ) { - c->msgid =MSG_TUR_TIMEOUT ; -return PATH_TIMEOUT ; - } - ct->nr_timeouts++; -/* -* Start a new thread while the old one is stalled. -* We have to prevent it from interfering with the new -* thread. We create a new context and leave the old -* one with the stale thread, hoping it will clean up -* eventually. -*/ -condlog (3 ,"%d:%d : tur thread not responding" , -major (ct->devt),minor (ct->devt)); - -/* -* libcheck_init will replace c->context. -* It fails only in OOM situations. In this case, return -* PATH_UNCHECKED to avoid prematurely failing the path. -*/ -if (libcheck_init (c) !=0 ) { - c->msgid =MSG_TUR_FAILED ; -return PATH_UNCHECKED ; - } - ((struct tur_checker_context *)c->context)->nr_timeouts = ct->nr_timeouts; - -if (!uatomic_sub_return (&ct->holders,1 )) { -/* It did terminate, eventually */ -cleanup_context (ct); - ((struct tur_checker_context *)c->context)->nr_timeouts =0 ; - } - - ct = c->context; - }else - ct->nr_timeouts =0 ; -/* Start new TUR checker */ -pthread_mutex_lock (&ct->lock); - tur_status = ct->state =PATH_PENDING ; - c->msgid = ct->msgid =MSG_TUR_RUNNING ; -pthread_mutex_unlock (&ct->lock); - ct->fd = c->fd; - ct->timeout = c->timeout; - ct->checked_state = false; -uatomic_add (&ct->holders,1 ); -uatomic_set (&ct->running,1 ); -tur_set_async_timeout (c); -setup_thread_attr (&attr,32 *1024 ,1 ); - r =start_checker_thread (&ct->thread, &attr, &ct->ctx); -pthread_attr_destroy (&attr); -if (r) { -uatomic_sub (&ct->holders,1 ); -uatomic_set (&ct->running,0 ); - ct->thread =0 ; -condlog (3 ,"%d:%d : failed to start tur thread, using" -" sync mode" ,major (ct->devt),minor (ct->devt)); -return tur_check (c->fd, c->timeout, &c->msgid); - } +/* Handle the case that the checker just completed */ +if (ct->rtx) { +check_runner_state (ct); + c->msgid = ct->tdata.msgid; +return ct->tdata.state; } -return tur_status; +/* create new checker thread */ + ct->tdata.fd = c->fd; + ct->tdata.timeout = c->timeout; + + ct->tdata.state =PATH_PENDING ; + ct->tdata.msgid =MSG_TUR_RUNNING ; +condlog (3 ,"%d:%d : starting checker" ,major (ct->tdata.devt), +minor (ct->tdata.devt)); + ct->rtx =get_runner (runner_callback, &ct->tdata,sizeof (ct->tdata), +1000000 * c->timeout); + +if (ct->rtx) { + c->msgid = ct->tdata.msgid; +return ct->tdata.state; + }else { +condlog (3 ,"%d:%d : failed to start tur thread, using sync mode" , +major (ct->tdata.devt),minor (ct->tdata.devt)); +return tur_check (c->fd, c->timeout, &c->msgid); + } }
Signature · 1 line
2.53.0
PATCH v3 5/5 libmultipath: tur checker: improve tur_deep_sleep() test
In reply to Martin WilckThe tur_deep_sleep() test serves to test the behavior of the tur checker when checkers time out and cannot be cancelled.
Up to now the checker timed out on every TUR_SLEEP_INTERVALth invocation. That made it impossible to test the MAX_TIMEOUTS logic, which requires multiple consecutive timeouts. To fix it, invert the logic, so that the checker just succeeds on every TUR_SLEEP_INTERVALth call.
Also, decrease loglevel of the messages from tur_deep_sleep().
--- libmultipath/checkers/tur.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
libmultipath/checkers/tur.c +3 -3
@@ -214,15 +214,15 @@ static void tur_deep_sleep(const struct tur_data *tdata)int oldstate;if (tdata->devt !=makedev (TUR_TEST_MAJOR ,TUR_TEST_MINOR ) || - ++sleep_cnt %TUR_SLEEP_INTERVAL !=0 ) + ++sleep_cnt %TUR_SLEEP_INTERVAL ==0 )return ; -condlog (1 ,"tur thread going to sleep for %ld seconds" , ts.tv_sec); +condlog (3 ,"tur thread going to sleep for %ld seconds" , ts.tv_sec);if (pthread_setcancelstate (PTHREAD_CANCEL_DISABLE , &oldstate) !=0 )condlog (0 ,"pthread_setcancelstate: %m" );if (nanosleep (&ts,NULL ) !=0 )condlog (0 ,"nanosleep: %m" ); -condlog (1 ,"tur zombie thread woke up" ); +condlog (3 ,"tur zombie thread woke up" );if (pthread_setcancelstate (oldstate,NULL ) !=0 )condlog (0 ,"pthread_setcancelstate (2): %m" );pthread_testcancel ();
Signature · 1 line
2.53.0