Testing absence of ticks with nohz_full

My company (Enea) wants to create test case(s) that verifies that you can run an application without having ticks going on the CPU that the application runs on. This would utilize the new nohz_full feature available from kernel version 3.10.

I cannot find any existing test case in LTP in this area, so I assume I have to create the test case myself.

One catch is that the current nohz_full implementation actually generates 1Hz. So part of the test could be to verify that we actually get 1Hz rather than full tick speed. It would also be possible to support the experimental patch from Kevin Hilman to actually turn ticks off for real. The test code code check whether the patch is applied and based on it expect 0Hz or 1Hz ticks.

Would LTP be a good place for such tests?

My company (Enea) wants to create test case(s) that verifies that you can run an application without having ticks going on the CPU that the application runs on. This would utilize the new nohz_full feature available from kernel version 3.10.

I cannot find any existing test case in LTP in this area, so I assume I have to create the test case myself.

One catch is that the current nohz_full implementation actually generates 1Hz. So part of the test could be to verify that we actually get 1Hz rather than full tick speed. It would also be possible to support the experimental patch from Kevin Hilman to actually turn ticks off for real. The test code code check whether the patch is applied and based on it expect 0Hz or 1Hz ticks.

Would LTP be a good place for such tests?

Regards
Mats Liljegren

On both cases you want to use trace events. So I'd rather suggest you to use perf tools or trace-cmd.

I believe that trace-cmd unfortunately uses per CPU periodic wake up on live recording CPUs so perhaps it's not a good candidate to test isolation. Too bad because kernelshark is cool for an overview.

Also perf supports some script languages for post processing: python, perl, etc... I think trace-cmd too but rather for custom display of events.

Adding a few more people in Cc.

Quoted 3 lines · On Mon, 10 Mar 2014 16:17:48 +0100 Frederic Weisbecker <[email protected]>…

On both cases you want to use trace events. So I'd rather suggest you to use perf tools or trace-cmd.

Quoted 3 lines

I believe that trace-cmd unfortunately uses per CPU periodic wake up on live recording CPUs so perhaps it's not a good candidate to test isolation. Too bad because kernelshark is cool for an overview.

Actually, it's not per-cpu. There's a thread per-cpu, but they do not need to run on the CPU they monitor. Heck, you can have all threads on just one CPU. Actually, they are not threads, they are forked children, but you get the idea.

Now that I have wakeup working (with help of irq_work), I should also be able to have them only wake if there's something to trace.

Quoted 4 lines

Also perf supports some script languages for post processing: python, perl, etc... I think trace-cmd too but rather for custom display of events.

Yeah, you can write python plugins that will change the display of the output. I'm not sure about full integrating with python though.

Quoted 4 lines · Mats Liljegren

My company (Enea) wants to create test case(s) that verifies that you can run an application without having ticks going on the CPU that the application runs on. This would utilize the new nohz_full feature available from kernel version 3.10.

Ah nice!

Quoted 11 lines

I cannot find any existing test case in LTP in this area, so I assume I have to create the test case myself.

One catch is that the current nohz_full implementation actually generates 1Hz. So part of the test could be to verify that we actually get 1Hz rather than full tick speed. It would also be possible to support the experimental patch from Kevin Hilman to actually turn ticks off for real. The test code code check whether the patch is applied and based on it expect 0Hz or 1Hz ticks.

Quoted 1 line

Would LTP be a good place for such tests?

If that can be of any help, you can have a look at this:
https://git.kernel.org/cgit/linux/kernel/git/frederic/dynticks-testing.git/

It's selftest that setup some isolation configuration, run a userspace loop for a few seconds and records various trace events like timers, workqueue, etc...

So it's not automatically telling if a CPU runs full dynticks, some analysis from a user on the traces is required.

Now it's fairly possible to write a small test that tells if the CPU noise is reduced to strict 1 Hz timer and nothing else. But that alone wouldn't provide safe guarantees because workqueues of whatever timer may happen anytime.

In really depends on your usecase. Do you want your CPU not be disturbed in order to maximize its throughput? That's the HPC case. There you can cope with a few noise from time to time. Then such a test would be relevant.

Now if you have latency requirement, which means having a tick or workqueue or anything disturbing you CPU would screw up everything then you're rather on the side of real time and then you need guarantee. For such a case I'd rather suggest live monitoring of trace events.

So there is two possibilities:

1) Show that the CPU runs in full dynticks for a given timeslice.

2) Monitoring events through the whole timeslices.

On both cases you want to use trace events. So I'd rather suggest you to use perf tools or trace-cmd.

I believe that trace-cmd unfortunately uses per CPU periodic wake up on live recording CPUs so perhaps it's not a good candidate to test isolation. Too bad because kernelshark is cool for an overview.

Also perf supports some script languages for post processing: python, perl, etc... I think trace-cmd too but rather for custom display of events.

I have given the "correct" way of counting ticks some thought, but I haven't come to a good conclusion. My current best bet is using function tracing on scheduler_tick(). It seems to be the most configuration and architecture independent way. But I'm not entirely sure it really is just one call for each tick. If you have better suggestions, I'd love to hear about it.

Quoted 4 lines

I believe that trace-cmd unfortunately uses per CPU periodic wake up on live recording CPUs so perhaps it's not a good candidate to test isolation. Too bad because kernelshark is cool for an overview.

I've invented a "count_ticks" script which hides ftrace complexity on other interesting ways than trace_cmd does. It logs calls to scheduler_tick during a command execution, and goes through this log afterwards counting the calls.

Quoted 2 lines

Also perf supports some script languages for post processing: python, perl, etc... I think trace-cmd too but rather for custom display of events.

While debugging is nice too, my first step is to just get something that can say PASS/FAIL, with some information about why it failed (e.g. because x ticks occurred that was not expected). Nothing more fancy than that.

My company (Enea) wants to create test case(s) that verifies that you can run an application without having ticks going on the CPU that the application runs on. This would utilize the new nohz_full feature available from kernel version 3.10.

I cannot find any existing test case in LTP in this area, so I assume I have to create the test case myself.

My company (Enea) wants to create test case(s) that verifies that you can run an application without having ticks going on the CPU that the application runs on. This would utilize the new nohz_full feature available from kernel version 3.10.

I cannot find any existing test case in LTP in this area, so I assume I have to create the test case myself.

testcases/kernel/partrt/nohz_full/Makefile new file +5 -0

+LDLIBS			+= -lrt
+
+MAKE_TARGETS		:= test_partrt_nohz_full
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +27 -0

new file mode 100644
index 0000000..750ab45
@@ -0,0 +1,636 @@
+/*
+ * Copyright (C) 2014, Enea AB.
+ *
+ * This program is free software;  you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ * the GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program;  if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ */
+
+#include <errno.h>
+#include <string.h>
+#include <getopt.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <sys/syscall.h>
+#include <sys/stat.h>
+#include <stdarg.h>
+#include <inttypes.h>

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +21 -0

+/* Used for RT load */
+volatile int dummy_value;
+
+/* When true (1), all children will terminate */
+static volatile int time_is_up = 0;
+
+/* Verbosity level chosen from program command line.
+ * 0 = Only errors and status reported
+ * 1 = Show info() messages
+ * 2 = Request verbose information from called sub-programs
+ * 3 = Request more verbose inormation from called sub-programs.
+ */
+static int verbose = 0;
+
+/* Name of application as given in argv[0] */
+static const char *appname;
+
+/* RT load function prototype */
+typedef void (child_func)(void);
+
+/* Report error a'la printf() and terminate test */

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +26 -0

+/* Amount of stack for RT load threads */
+#define STACK_SZ (64 * 1024)
+
+static void cleanup(void);
+
+/*
+ * Check exit status value.
+ * If any signs of error is found, report error and abort testing.
+ */
+static void assert_exit_status(const char *cmd, int status)
+{
+	if (WIFSIGNALED(status))
+		err("%s: Child terminated unexpectedly due to signal nr %d",
+		    cmd,
+		    WTERMSIG(status));
+	if (!WIFEXITED(status))
+		err("%s: Child terminated unexpectedly",
+		    cmd);
+	if (WEXITSTATUS(status) != 0)
+		err("%s: Child exited with exit status %d",
+		    cmd,
+		    WEXITSTATUS(status));
+}
+
+/*
+ * Generate a load that is guaranteed not to provoke ticks.

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +31 -0

+/*
+ * Execute command, supplying data to the commands stdin using infp if infp
+ * is non-NULL, and supplying data output from the commands stdout using
+ * outfp if outfp is non-NULL.
+ * The command returns the pid of the command executed.
+ */
+static pid_t
+pid_popen(const char *cmd, FILE **infp, FILE **outfp)
+{
+	int p_stdin[2], p_stdout[2];
+	pid_t pid;
+
+	if (((infp != NULL) && (pipe(p_stdin) != 0)) ||
+		((outfp != NULL) && (pipe(p_stdout) != 0)))
+		err("%s: Failed executing command: pipe(): %s",
+			cmd, strerror(errno));
+
+	pid = fork();
+
+	if (pid < 0)
+		err("%s: Failed executing command: fork(): %s",
+			cmd, strerror(errno));
+
+	if (pid == 0) {
+		if ((infp != NULL) && (dup2(p_stdin[READ], READ) == -1))
+			err("%s: pid %lu: dup() to stdin failed: %s",
+				cmd, (unsigned long) pid, strerror(errno));
+		if ((outfp != NULL) && (dup2(p_stdout[WRITE], WRITE) == -1))
+			err("%s: pid %lu: dup() to stdout failed: %s",
+				cmd, (unsigned long) pid, strerror(errno));
+

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +17 -0

+		execl("/bin/sh", "sh", "-c", cmd, NULL);
+
+		err("%s: pid %lu: exec() error: %s",
+			cmd, (unsigned long) pid, strerror(errno));
+	}
+
+	if (((infp != NULL) && (close(p_stdin[READ]) == -1)) ||
+		((outfp != NULL) && (close(p_stdout[WRITE]) == -1)))
+		err("%s: pid %lu: Parent: close() of pipe failed: %s",
+			cmd, (unsigned long) pid, strerror(errno));
+
+	if (infp != NULL) {
+		(*infp) = fdopen(p_stdin[WRITE], "w");
+		if (*infp == NULL)
+			err("%s: pid %lu: fdopen() of stdin failed: %s",
+				cmd, (unsigned long) pid, strerror(errno));
+	}

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +4 -0

+/*
+ * Wait for a command with given pid to finish, and assert that the command
+ * returned success (0).
+ */

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +30 -0

+}
+
+/*
+ * Execute given command using shell, ensure success (0) is returned.
+ */
+static void shell(const char *cmd, ...)
+{
+	char *cmd_buf;
+	va_list va;
+
+	va_start(va, cmd);
+	if (vasprintf(&cmd_buf, cmd, va) == -1)
+		err("%s: Not valid printf format, or out of memory", cmd);
+	va_end(va);
+
+	info("%s: Executing", cmd_buf);
+	pid_wait(pid_popen(cmd_buf, NULL, NULL));
+	info("%s: Returns success", cmd_buf);
+
+	free(cmd_buf);
+}
+
+/*
+ * Execute given command using shell, ensure success (0) is returned.
+ * Only reports errors.
+ */
+static void shell_silent(const char *cmd, ...)
+{
+	char *cmd_buf;
+	va_list va;

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +11 -0

+/*
+ * Convert string to unsigned long.
+ * Error prefix expressed as string.
+ */
+static unsigned long str_to_ulong(
+	const char *str,
+	const char *err_prefix,
+	int base
+	)
+{
+	char *endptr;

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +40 -0

+/*
+ * Call a command using shell.
+ * Command line expressed as va_list.
+ * Returns the command's first line of output.
+ */
+static char * shell_str(char *dest, int size, const char *cmd, ...)
+{
+	int len;
+	char *cmd_buf;
+	FILE *file;
+	pid_t pid;
+	va_list va;
+
+	/* Build command line */
+	va_start(va, cmd);
+	if (vasprintf(&cmd_buf, cmd, va) == -1)
+		err("%s: Not valid printf format, or out of memory", cmd);
+	va_end(va);
+
+	/* Launch command */
+	pid = pid_popen(cmd_buf, NULL, &file);
+	info("%s: Executing, pid %lu", cmd_buf, (unsigned long) pid);
+
+	/* Read commands stdout */
+	if (fgets(dest, size, file) == NULL) {
+		if (feof(file))
+			err("%s: pid %lu: Expected output from the command, but got nothing",
+				cmd_buf, (unsigned long) pid);
+		else
+			err("%s: pid %lu: Could not read command output: %s",
+				cmd_buf, (unsigned long) pid, strerror(errno));
+	}
+
+	/* Get rid of terminating newline, if any */
+	len = strlen(dest);
+	if (dest[len-1] == '\n')
+		dest[len-1] = '\0';
+
+	/* Wait until command execution finish.
+	 * Main reason for doing this even though we've got what we want is for

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +13 -0

+/*
+ * Call a command using shell.
+ * Command line expressed similar to printf().
+ * Expects that the command outputs a value as the first line of output
+ * from stdout.
+ * Asserts that the command returned success (0).
+ */
+static unsigned long shell_int(int base, const char *cmd, ...)
+{
+	va_list va;
+	unsigned long val;
+	char val_str[64];
+	char *cmd_buf;

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +23 -0

+/*
+ * Struct used for synchronization between child_entry() and launch_child().
+ */
+struct child_params {
+	/* Function to run */
+	child_func *func;
+
+	/* cpuset partition to run in */
+	const char *cpu_partition;
+
+	/* CPU ID to run on. Must be one of the CPUs in chosen cpu_partition */
+	int cpu;
+
+	/* Written by child_entry() when no more activities that might provoke
+	 * ticks are expected. */
+	volatile int started;
+};
+
+/*
+ * Child tramponline function for clone() function.
+ */
+static int child_entry(void *arg)
+{

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +7 -0

+	/* Signal parent that this child now enters RT load function
+	 * (i.e. no more ticks are expected) */
+	child_params->started = 1;
+
+	child_params->func();
+	return 0;
+}

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +10 -0

+	pid_t pid;
+	int status;
+
+	if (!__sync_bool_compare_and_swap(&cleanup_entered, 0, 1,
+						cleanup_entered)) {
+		info("Cleanup: Already cleaning, exiting");
+		return;
+	}
+
+	info("Cleanup: Terminating children");

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +4 -0

+			info("Cleanup: %s: Has terminated successfully", cmd);
+		}
+	} while (pid != -1);
+

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +28 -0

+	char buf[256] = {0};
+	char *curr;
+	int range_first;
+	int range_last;
+	unsigned long mask = 0;
+
+	if (file == -1)
+		err("%s: Open failed, have you run 'partrt create'?",
+			nohz_cpus_filename);
+
+	SAFE_READ(cleanup, 0, file, buf, sizeof (buf) - 1);
+
+	curr = buf;
+	while (*curr != '\0') {
+		char *curr_next;
+		int bit;
+
+		range_first = range_last = strtoull(curr, &curr_next, 10);
+		if (curr == curr_next)
+			err("%s: %s: Cannot parse CPU list at %s",
+				nohz_cpus_filename, buf, curr);
+		curr = curr_next;
+		if (*curr == '-') {
+			curr++;
+			range_last = strtoull(curr, &curr_next, 10);
+			if (curr == curr_next)
+				err("%s: %s: Cannot parse CPU list at %s",
+					nohz_cpus_filename, buf, curr);

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +17 -0

+	}
+
+	return mask;
+}
+
+/*
+ * Prepare for test.
+ * Perform CPU partitioning and start RT load.
+ */
+static int setup(void)
+{
+	const unsigned long nohz_mask = determine_nohz_mask();
+	int bit;
+	int nr_children = 0;
+
+	info("Nohz CPU mask: %#lx", nohz_mask);
+	tst_require_root(NULL);

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +32 -0

+	nr_ticks = shell_int(0, "count_ticks --cpu rt --batch --end");
+
+	if (expect_0hz) {
+		if (nr_ticks != 0)
+			fail("Expected no ticks, but got %" PRIu64, nr_ticks);
+		else
+			pass("No ticks occurred");
+	} else {
+		if (nr_ticks > expected_ticks)
+			fail("Expected maximum %" PRIu64 " ticks, but got %" PRIu64,
+			     expected_ticks, nr_ticks);
+		else
+			pass("%" PRIu64 " ticks occurred, which was expected",
+			     nr_ticks);
+	}
+
+	cleanup();
+	tst_exit();
+}
+
+/*
+ * Show usage text and exit.
+ */
+static void usage(void)
+{
+	printf(
+		"Usage: %s [options]\n"
+		"       %s --help\n"
+		"Test whether a CPU can be isolated and made tickless even under load.\n"
+		"\n"
+		"  -h, --help         Display this usage text and exit.\n"
+		"  -d, --duration     Number of seconds to run the test.\n"

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +32 -0

+int main(int argc, char *argv[])
+{
+	time_t duration = 0;
+	int done = 0;
+
+	static const struct option options[] = {
+		{"verbose", optional_argument, NULL, 'v'},
+		{"duration", required_argument, NULL, 'd'},
+		{"help", no_argument, NULL, 'h'}
+	};
+
+	static const char short_options[] = "v::d:p:h";
+
+	appname = basename(argv[0]);
+
+	while (!done) {
+		const int opt = getopt_long(
+			argc, argv, short_options, options, NULL);
+
+		switch (opt) {
+		case -1:
+			done = 1;
+			break;
+		case '?':
+			err("Use '--help' for usage");
+		case 'h':
+			usage();
+		case 'd':
+			duration =
+				(time_t) str_to_ulong(optarg, "--duration", 0);
+			break;
+		case 'v':

As for -Werror, I can remove it, but not without shedding some tears... This was my way of assuring that no one who contributed to my code will be able to slip in warnings unnoticed. To turn things around: The -Werror is a way of saying "Now this code has once compiled without warnings, and from now on it is not ok to introduce new warnings".

You might think that it is never ok to introduce warnings, but when I compile I get some warnings. I didn't want this to happen to my code as well.

Quoted 2 lines

No useless comments like this please (see paragraph 1.4 in Test Writing Guidelines).

Hmm, must have been some copy and paste. I'm usually bad at producing comments... ;-)

Quoted 29 lines

Quoted 23 lines

+/*
+ * Call a command using shell.
+ * Command line expressed similar to printf().
+ * Expects that the command outputs a value as the first line of output
+ * from stdout.
+ * Asserts that the command returned success (0).
+ */

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +10 -0

+static unsigned long shell_int(int base, const char *cmd, ...)
+{
+	va_list va;
+	unsigned long val;
+	char val_str[64];
+	char *cmd_buf;
+
+	va_start(va, cmd);
+	if (vasprintf(&cmd_buf, cmd, va) == -1)
+		err("%s: Not valid printf format, or out of

memory", cmd);

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +3 -0

+	va_end(va);
+	val = str_to_ulong(
+		shell_str(val_str, sizeof (val_str), "%s", cmd),

cmd, base); +

Review

Quoted 2 lines

(quoted diff) +2 -0

+	return val;
+}

The cmd_buf is not used nor freed, the additional parameters are ignored etc.

Quoted 4 lines

If you are going to write shell_int() by using shell_str you must create vashell_str() first and build both shell_str and shell_int on the top of it. But given that this function is used only once I would strongy sugest to use shell_str() and SAFE_STRTOUL() in the caller.

I agree. That would simplify things. Sometimes my ambition to make things general get a bit out of hand...

Quoted 4 lines

Quoted 4 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +4 -0

+{
+	char *stack = SAFE_MALLOC(cleanup, STACK_SZ);
+	int tid;
+	struct child_params child_params = {

Quoted 8 lines

Quoted 5 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +4 -0

+		func, cpu_partition, cpu, 0
+	};
+
+	tid = ltp_clone(CLONE_VM, child_entry, &child_params,

STACK_SZ, stack);

Is there any reason to use clone instead of fork() of pthread_create()?

Yep, fork() will produce a process, but I want a thread. But I also wanted the child PID, which is why I didn't use pthread_create().

Quoted 7 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c line 395

Quoted 2 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +2 -0

+	if (tid == -1)
+		err ("clone(): %s", strerror(errno));

^
no spaces between function names and brackets

Have you used checkpatch.pl as suggested in the Test Writing Guidelines?

No, but that was my intention before sending the "real" patch. This current round was RFC since I wanted to know if this is something that could be upstreamed before I do the final polishing on it.

Quoted 9 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c line 399

Quoted 4 lines

(quoted diff) +1 -0

+	info("tid %d: Starting %s load on cpu %d", tid,

cpu_partition, cpu);

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +2 -0

+	while (!child_params.started)
+		sched_yield();

I do not like the bussy loop here much (We have a pipe base child parent synchronization code in the LTP library, see paragraph 2.2.7 in the Test Writing Guidelines) but given that all the childs do is bussy loop...

I'm not sure why this is a problem. Using shared memory seems like a simpler design that using FIFO dependent on sharing the same current working directory. I also wonder whether tst_checkpoint.h is dependent on having different processes rather than different threads to synchronize.

The whole point with this is to make sure the children are up and running (i.e. no shell commands left to be done at startup). And this was the simplest design I could think of to ensure this.

Quoted 10 lines

Quoted 10 lines

(quoted diff) +1 -0

+			info("Cleanup: %s: Has terminated

successfully", cmd);

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +8 -0

+		}
+	} while (pid != -1);
+
+	if (errno != ECHILD)
+		err("Cleanup: wait() failed: %s", strerror(errno));
+
+	info("Cleanup: Done");
+}

Quoted 8 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c line 445

Quoted 6 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +4 -0

+
+static unsigned long determine_nohz_mask(void)
+{
+	const char *nohz_cpus_filename =

"/sys/fs/cgroup/cpuset/rt/cpuset.cpus";

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +1 -0

+	int file = open(nohz_cpus_filename, O_RDONLY);

this is not file but fd (file descriptor)

Where the fd represents a file... Whatever. I can change the name.

Quoted 10 lines

Quoted 10 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +9 -0

+/*
+ * Test main function.
+ * Perform tick measurement for the given number of seconds.
+ */
+static void test(time_t duration)
+{
+	uint64_t nr_ticks;
+	time_t time_finished;
+	/* If sched_tick_max_deferment patch has been applied,

expect that the

Quoted 8 lines

Quoted 6 lines

(quoted diff) +1 -0

+	 * partitioning has disabled ticks completely. Otherwise,

expect

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +3 -0

+	 * 1Hz ticks */
+	const int expect_0hz =
+

access("/sys/kernel/debug/sched_tick_max_deferment", F_OK) == 0;

This is a bit hacky, I would just use if (...) instead.

I like const declaration, they make things less error prone. So personally I prefer small amount of hackiness to buy me less error prone code... But this is a matter of taste.

Quoted 7 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c line 527

Quoted 3 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +3 -0

+	const time_t current_time = time(NULL);
+
+	const unsigned int nr_children = setup();

Returning number of children from setup is a bit confusing. Maybe it would be better to make nr_children global variable or rename the function.

The name "setup" doesn't really give any hints on what is returned, only what is being done. I'm not confused by it, but I guess I'm biased...

Would you prefer that "setup" is renamed to "setup_return_nr_children"?

Quoted 10 lines

Quoted 10 lines

+/*
+ * Wait for a command with given pid to finish, and assert that the command
+ * returned success (0).
+ */

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +5 -0

+static void
+pid_wait(pid_t pid)
+{
+	int status;
+	char prefix[64];

Quoted 9 lines

Quoted 6 lines

(quoted diff) +2 -0

+
+	snprintf(prefix, sizeof (prefix), "Pid %lu", (unsigned

long) pid); +

(quoted diff) +2 -0

+	if (wait(&status) == -1)
+		err("%s: wait() failed: %s", prefix,

strerror(errno));

Use SAFE_WAIT() instead.

I couldn't find that function in include directory. Did I search in the wrong place?

Quoted 12 lines

Quoted 12 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +10 -0

+	const char *str,
+	const char *err_prefix,
+	int base
+	)
+{
+	char *endptr;
+	unsigned long val;
+	val = strtoull(str, &endptr, base);
+	if (endptr == str)
+		err("%s: %s: Expected unsigned decimal value",

err_prefix, str);

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +1 -0

+	if (*endptr != '\0')

Quoted 8 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c line 258

Quoted 6 lines

(quoted diff) +1 -0

+		err("%s: %s: Garbage character '%c' found after

decimal value",

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +4 -0

+			err_prefix, str, *endptr);
+
+	return val;
+}

Use SAFE_STRTOUL() instead.

That would loose a lot of information useful for figuring out what the error is. For example, when called for translating command line options to integers, using SAFE_STRTOUL() will only say that "this string is not an integer" without giving any hint that we are talking about a command line option, and which option that is being parsed.

Quoted 16 lines

Quoted 16 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +16 -0

+	pid_t pid;
+	int status;
+
+	if (!__sync_bool_compare_and_swap(&cleanup_entered, 0, 1,
+						cleanup_entered)) {
+		info("Cleanup: Already cleaning, exiting");
+		return;
+	}
+
+	info("Cleanup: Terminating children");
+
+	time_is_up = 1;
+
+	do {
+		pid = wait(&status);
+		if (pid != -1) {

Quoted 8 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c line 431

Quoted 4 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +4 -0

+			char cmd[64];
+			snprintf(cmd, sizeof (cmd), "Pid %lu",
+				 (unsigned long) pid);
+			assert_exit_status(cmd, status);

I do not understand why you pass "Pid %lu" string instead of the pid. It's more messy this way.

This is because assert_exit_status() sometimes checks result of executed commands, and in those cases this function gets a command line rather than PID.

Quoted 6 lines

Quoted 2 lines

I couldn't find that function in include directory. Did I search in the wrong place?

Ah, you are right, that one is missing, sorry.

I will add it.

Ok.

This patch introduces a new test case, partrt_nohz_full.

This is sent as an RFC since as far as I understand it ltp is under stabilization. But I still wanted to send what I've got currently to see if there are any major issues with it.

This test case requires a couple of tools which can be cloned from here:

https://github.com/OpenEneaLinux/rt-tools

The tools are scripts so they are easily copied to somewhere PATH will find them.

The patch applies on ltp, which can be cloned from here:

https://github.com/linux-test-project/ltp.git

The patch applies to commit hash b24ce8f6bcf315078bdc48bbddf780b964ffa59a.

To run the test case, do the following:

partrt create <cpumask>

runltp -f partrt_nohz_full

where <cpumask> should be a hexadecimal mask of nohz_full kernel boot parameter. If you're too lazy to translate it by hand, you can use list2mask tool from the rt-tools repository:

partrt create 0x$(list2mask --nohz)
runltp -f partrt_nohz_full

I've thought about this. The problem is that these tools is only one way of doing it. Performing CPU isolation in Linux is still something of a black art, so my idea was to show one way that we have got it working for at least some boards. Since how to do this is also dependent on architecture, board, kernel version and kernel configuration I didn't want to tie these script too closely to the test case since the test case by itself is quite generic.

Having these tools as copies in LTP repository gives additional maintenance burden that I feel is unnecessary. The tools are needed for other purposes as well, so moving them to LTP is not an option.

But if you have a good idea how to solve this dilemma, I'd like to hear about it.

testcases/kernel/partrt/README new file +39 -0

+The following build time kernel configurations must be enabled:
+
+        CONFIG_NO_HZ_FULL=y
+        CONFIG_CPUSETS=y
+
+These kernel configuration parameters are also documented in the
+READ.kernel_config file in LTP root directory.
+
+The following kernel boot parameter needs to be set:
+
+        nohz_full=<cpu list>
+
+There might be other parameters needed to actually get nohz_full working, but
+this is largely dependent on architecture and kernel version.
+
+How to run
+----------
+
+The partrt_nohz_full test is run by runltp like so:
+
+        ./runltp -f partrt_nohz_full
+
+To make life interesting, you can play with "-n" and "-i" options to runltp, to
+generate some load in the non-realtime partition:
+
+        ./runltp -n -i 5 -f partrt_nohz_full
+
+To get some more help with the cause of a failing test, do this as root before
+running the test:
+
+        echo 1 > /sys/kernel/debug/tracing/events/enable
+
+and then look at trace after test failed:
+
+        cat /sys/kernel/debug/tracing/trace
+
+Since "count_ticks" also uses ftrace and will update tracing_cpumask to only
+trace the nohz_full CPUs, the trace should be close to empty if the run was
+successful.

testcases/kernel/partrt/nohz_full/.gitignore new file +1 -0

new file mode 100644
index 0000000..5848b21
@@ -0,0 +1 @@
+/test_partrt_nohz_full

testcases/kernel/partrt/nohz_full/Makefile new file +23 -0

new file mode 100644
index 0000000..5528dc0
@@ -0,0 +1,23 @@
+#
+#  Copyright (C) 2014, Enea AB.
+#
+#  This program is free software;  you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation; either version 2 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY;  without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+#  the GNU General Public License for more details.
+#
+
+top_srcdir		?= ../../../..
+
+include $(top_srcdir)/include/mk/testcases.mk
+
+LDLIBS			+= -lrt
+
+MAKE_TARGETS		:= test_partrt_nohz_full
+
+include $(top_srcdir)/include/mk/generic_leaf_target.mk

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +41 -0

new file mode 100644
index 0000000..1343b71
@@ -0,0 +1,551 @@
+/*
+ * Copyright (C) 2014, Enea AB.
+ *
+ * This program is free software;  you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ * the GNU General Public License for more details.
+ */
+
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <string.h>
+#include <getopt.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <sys/syscall.h>
+#include <sys/stat.h>
+#include <sys/timerfd.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <limits.h>
+#include <time.h>
+#include <sched.h>
+#include <fcntl.h>
+
+#include <test.h>
+#include <usctest.h>
+#include <safe_macros.h>
+#include <safe_stdio.h>
+
+const char *TCID = "partrt_nohz_full";
+const int TST_TOTAL = 1;
+
+/* Used for RT load */
+volatile int dummy_value;

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +36 -0

+/* Name of application as given in argv[0] */
+static const char *appname;
+
+/* Boolean describing whether RT partition in cpuset needs to be created */
+int need_partrt_create = 0;
+
+/* RT load function prototype */
+typedef void (child_func)(void);
+
+/* Amount of stack for RT load threads */
+#define STACK_SZ (64 * 1024)
+
+/* Default path to cgroups/cpuset */
+#define CPUSET_ROOT "/sys/fs/cgroup/cpuset"
+
+/* Expected path to cpuset RT partition */
+#define CPUSET_RT_PATH CPUSET_ROOT "/rt"
+
+/* Expected path to RT partition tasks list file */
+#define CPUSET_RT_TASKS_FILE CPUSET_RT_PATH "/tasks"
+
+/* Expected path to RT partition CPU list file */
+#define CPUSET_RT_CPUS_FILE CPUSET_RT_PATH "/cpuset.cpus"
+
+/* File to look for to determine whether true 0Hz can be achieved */
+#define SCHED_TICK_MAX_FILE "/sys/kernel/debug/sched_tick_max_deferment"
+
+/* File containing a list of filesystems supported by the kernel */
+#define PROC_FILESYSTEMS_FILE "/proc/filesystems"
+
+static void cleanup(void);
+
+/*
+ * Check exit status value.
+ * If any signs of error is found, report error and abort testing.
+ */

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +28 -0

+	if (!WIFEXITED(status))
+		tst_brkm(TBROK, cleanup, "%s: Child terminated unexpectedly",
+			 cmd);
+	if (WEXITSTATUS(status) != 0)
+		tst_brkm(TBROK, cleanup, "%s: Child exited with exit status %d",
+			 cmd,
+			 WEXITSTATUS(status));
+}
+
+/*
+ * Execute given command using shell, ensure success (0) is returned.
+ */
+static void shell(const char *cmd, ...)
+{
+	char *cmd_buf;
+	va_list va;
+
+	va_start(va, cmd);
+	if (vasprintf(&cmd_buf, cmd, va) == -1)
+		tst_brkm(TBROK, cleanup,
+			 "%s: Not valid printf format, or out of memory", cmd);
+	va_end(va);
+
+	tst_resm(TINFO, "%s: Executing", cmd_buf);
+	assert_exit_status(cmd_buf, system(cmd_buf));
+	tst_resm(TINFO, "%s: Returns success", cmd_buf);
+
+	free(cmd_buf);

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +36 -0

+{
+	char *endptr;
+	unsigned long val;
+	val = strtoull(str, &endptr, 0);
+	if (endptr == str)
+		tst_brkm(TBROK, cleanup,
+			 "%s: %s: Expected unsigned decimal value",
+			 err_prefix, str);
+	if (*endptr != '\0')
+		tst_brkm(TBROK, cleanup,
+			 "%s: %s: Garbage character '%c' found after decimal value",
+			 err_prefix, str, *endptr);
+
+	return val;
+}
+
+/*
+ * Call a command using shell.
+ * Command line expressed as va_list.
+ * Returns the command's first line of output.
+ */
+static char *shell_str(char *dest, int size, const char *cmd, ...)
+{
+	int len;
+	char *cmd_buf;
+	FILE *file;
+	va_list va;
+
+	/* Build command line */
+	va_start(va, cmd);
+	if (vasprintf(&cmd_buf, cmd, va) == -1)
+		tst_brkm(TBROK, cleanup,
+			 "%s: Not valid printf format, or out of memory", cmd);
+	va_end(va);
+
+	/* Launch command */

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +56 -0

+	tst_resm(TINFO, "%s: Executing", cmd_buf);
+
+	/* Read commands stdout */
+	if (fgets(dest, size, file) == NULL) {
+		if (feof(file))
+			tst_brkm(TBROK, cleanup,
+				 "%s: Expected output from the command, but got nothing",
+				 cmd_buf);
+		else
+			tst_brkm(TBROK | TERRNO, cleanup,
+				 "%s: Could not read command output",
+				 cmd_buf);
+	}
+
+	/* Get rid of terminating newline, if any */
+	len = strlen(dest);
+	if (dest[len-1] == '\n')
+		dest[len-1] = '\0';
+
+	/* Wait until command execution finish.
+	 * Main reason for doing this even though we've got what we want is for
+	 * better error detection. The alternative could be to let cleanup()
+	 * handle it if process hasn't finished by then. */
+	assert_exit_status(cmd_buf, pclose(file));
+
+	tst_resm(TINFO, "%s: Returns: %s", cmd_buf, dest);
+
+	free(cmd_buf);
+
+	return dest;
+}
+
+static int child_entry(void *arg)
+{
+	(void) arg;
+
+	while (!time_is_up)
+		dummy_value = rand();
+
+	return 0;
+}
+
+/*
+ * Start a new thread executing the given function, on the given cpuset
+ * partition and CPU ID. Note that the CPU ID must be one of the
+ * available CPUs in the cpuset partition.
+ */
+static void launch_child(int cpu)
+{
+	char *stack = SAFE_MALLOC(cleanup, STACK_SZ);
+	int tid;
+	const int cpuset_tasks_fd = SAFE_OPEN(cleanup, CPUSET_RT_TASKS_FILE,
+					      O_WRONLY);
+	char *tid_str;
+	cpu_set_t cpu_set;
+

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +14 -0

+
+	/* Move child to indicated CPU within the RT partition */
+	CPU_ZERO(&cpu_set);
+	CPU_SET(cpu, &cpu_set);
+	if (sched_setaffinity(tid, sizeof(cpu_set), &cpu_set) < 0)
+		tst_brkm(TBROK | TERRNO, cleanup,
+			"pid %u: sched_setaffinity() failed",
+			tid);
+
+	tst_resm(TINFO, "pid %d: Starting RT load on cpu %d",
+		 tid, cpu);
+}
+
+static void cleanup(void)

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +34 -0

+	tst_resm(TINFO, "Cleanup: Terminating children");
+
+	time_is_up = 1;
+
+	do {
+		pid = wait(&status);
+		if (pid != -1) {
+			char cmd[64];
+			snprintf(cmd, sizeof(cmd), "Pid %lu",
+				 (unsigned long) pid);
+			assert_exit_status(cmd, status);
+			tst_resm(TINFO,
+				 "Cleanup: %s: Has terminated successfully",
+				 cmd);
+		}
+	} while (pid != -1);
+
+	if (errno != ECHILD)
+		tst_brkm(TBROK | TERRNO, NULL, "Cleanup: wait() failed");
+
+	if (need_partrt_create)
+		shell("partrt undo");
+
+	tst_resm(TINFO, "Cleanup: Done");
+}
+
+static unsigned long determine_nohz_mask(void)
+{
+	int range_first;
+	int range_last;
+	int bit;
+	unsigned long mask = 0;
+	FILE *stream = SAFE_FOPEN(cleanup, CPUSET_RT_CPUS_FILE, "r");
+	int nr_matches;

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +47 -0

+	}
+
+	SAFE_FCLOSE(cleanup, stream);
+
+	if (nr_matches == -1)
+		tst_brkm(TBROK | TERRNO, cleanup, "%s: fscanf() failed",
+			 CPUSET_RT_CPUS_FILE);
+
+	return mask;
+}
+
+static void tool_available(char *tool_name, const char *env_path)
+{
+	int success = 0;
+
+	if (strchr(tool_name, '/') != NULL) {
+		if (access(tool_name, X_OK) == 0)
+			success = 1;
+	} else {
+		char full_path[PATH_MAX];
+		const char *curr;
+		char * const alloced_path = strdup(env_path);
+		char *next = alloced_path;
+
+		for (curr = strsep(&next, ":");
+		     curr != NULL;
+		     curr = strsep(&next, ":")) {
+			if (*curr == '\0')
+				curr = ".";
+			if (snprintf(full_path, sizeof(full_path),
+					"%s/%s", curr, tool_name
+					) >= (int)sizeof(full_path))
+				continue;
+			if (access(full_path, X_OK) == 0) {
+				success = 1;
+				break;
+			}
+		}
+		free(alloced_path);
+	}
+
+	if (!success)
+		tst_brkm(TCONF, cleanup,
+			"%s tool not found, skipping test. Use 'git submodule update utils/rt-tools' to include needed tools in the build.",
+			tool_name);
+}
+

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +5 -0

+}
+
+static void cpuset_check(void)
+{
+	FILE * const stream = SAFE_FOPEN(cleanup, PROC_FILESYSTEMS_FILE, "r");

*

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +66 -0

+	char *name;
+
+	while (fscanf(stream, "%as", &name) > 0) {
+		if (strcmp(name, "cpuset") == 0) {
+			free(name);
+			return;
+		}
+		free(name);
+	}
+
+	tst_brkm(TCONF, cleanup, "CPUSET not configured in kernel");
+}
+
+/*
+ * Setup and return number of child threads
+ */
+static int setup_children(void)
+{
+	unsigned long nohz_mask;
+	int bit;
+	int nr_children = 0;
+
+	time_is_up = 0;
+
+	tools_check();
+	tst_require_root(NULL);
+	cpuset_check();
+
+	if (access(CPUSET_RT_PATH, F_OK) == -1)
+		need_partrt_create = 1;
+
+	if (need_partrt_create)
+		shell("partrt create $(list2mask --nohz)");
+
+	nohz_mask = determine_nohz_mask();
+	tst_resm(TINFO, "Nohz CPU mask: %#lx", nohz_mask);
+
+	for (bit = 0; bit < (int) (sizeof(nohz_mask) * 8); bit++) {
+		if (nohz_mask & (1lu << bit)) {
+			launch_child(bit);
+			nr_children++;
+		}
+	}
+
+	tst_resm(TINFO, "All children started");
+	return nr_children;
+}
+
+/*
+ * Perform tick measurement for the given number of seconds.
+ */
+static void test(time_t duration, unsigned nr_children)
+{
+	uint64_t nr_ticks;
+	time_t time_finished;
+	/* If sched_tick_max_deferment patch has been applied, expect that the
+	 * partitioning has disabled ticks completely. Otherwise, expect
+	 * 1Hz ticks */
+	const int expect_0hz =
+		access(SCHED_TICK_MAX_FILE, F_OK) == 0;
+	const time_t current_time = time(NULL);
+
+	const uint64_t expected_ticks = expect_0hz ? 0 : duration * nr_children;
+	char val_str[64];
+	static const char count_ticks_end_cmd[] =
+		"count_ticks --cpu rt --batch --end";

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +38 -0

+	timer_fd = timerfd_create(CLOCK_MONOTONIC, 0);
+	timeout.it_value.tv_sec = duration;
+	timerfd_settime(timer_fd, 0, &timeout, NULL);
+
+	/* Wait for timeout */
+	SAFE_READ(cleanup, 1, timer_fd, &nr_timeouts, sizeof(nr_timeouts));
+	if (nr_timeouts != 1)
+		tst_brkm(TBROK, cleanup,
+			"Multiple timeouts when single timeout was requested");
+
+	shell_str(val_str, sizeof(val_str), count_ticks_end_cmd);
+	nr_ticks = str_to_ulong(val_str, count_ticks_end_cmd);
+
+	if (expect_0hz) {
+		if (nr_ticks != 0)
+			tst_resm(TFAIL, "Expected no ticks, but got %" PRIu64,
+				 nr_ticks);
+		else
+			tst_resm(TPASS, "No ticks occurred");
+	} else {
+		if (nr_ticks > expected_ticks)
+			tst_resm(TFAIL,
+				 "Expected maximum %" PRIu64
+				 " ticks, but got %" PRIu64,
+				 expected_ticks, nr_ticks);
+		else
+			tst_resm(TPASS,
+				 "%" PRIu64
+				 " ticks occurred, which was expected",
+				 nr_ticks);
+	}
+}
+
+static void usage(void)
+{
+	printf(
+		"Usage: %s [options]\n"
+		"       %s --help\n"

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +25 -0

+}
+
+int main(int argc, char *argv[])
+{
+	long duration = 0;
+	int lc;
+	unsigned int nr_children;
+	char *error_msg;
+	char *verbose_str = NULL;
+	char *duration_str = NULL;
+
+	const option_t options[] = {
+		{"v:", NULL, &verbose_str},
+		{"d:", NULL, &duration_str},
+		{NULL, NULL, NULL}
+	};
+
+	verbose = 0;
+	appname = basename(argv[0]);
+
+	error_msg = parse_opts(argc, argv, options, usage);
+	if (error_msg != NULL)
+		tst_brkm(TBROK, NULL, "Error parsing command line: %s",
+			error_msg);
+

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +1 -0

+}

utils/.gitignore +3 -0

index a582748..f63e51a 100644
@@ -48,3 +48,6 @@
 /sctp/func_tests/test_tcp_style_v6
 /sctp/func_tests/test_timetolive
 /sctp/func_tests/test_timetolive_v6
+/count_ticks
+/list2mask
+/partrt

utils/Makefile +0 -0

index 1508b35..8c2fc7e 100644

utils/Makefile +3 -0

 FFSBDIR			:= ffsb-6.0-rc2
 FILTER_OUT_DIRS		:= $(FFSBDIR)

@@ -35,6 +39,9 @@ $(FFSB): $(abs_srcdir)/$(FFSBDIR)
 ffsb: $(FFSB)
 	cp $(FFSB) ffsb
 
+partrt list2mask count_ticks:
+	cp rt-tools/install/bin/$@ $@
+
 trunk-all: $(FFSB)
 
 trunk-clean:: | ffsb-clean

utils/rt-tools new file +1 -0

new file mode 160000
index 0000000..f75b334
@@ -0,0 +1 @@
+Subproject commit f75b334922a2243d0b757c0627c6f1c8440818c0

Signature · 13 lines

1.7.10.4


------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.  Get 
unparalleled scalability from the best Selenium testing platform available.
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list

Quoted 6 lines

Quoted 6 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +6 -0

+	/* Launch command */
+	errno = 0;
+	file = popen(cmd_buf, "r");
+	if (file == NULL) {
+		if (errno == 0)
+			tst_brkm(TBROK, cleanup, "%s: popen(): Out

Quoted 8 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c line 182

Quoted 6 lines

of memory",

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +5 -0

+				 cmd_buf);
+		else
+			tst_brkm(TBROK | TERRNO, cleanup,
+				"%s: popen() failed", cmd_buf);
+	}

This may be worth of SAFE_POPEN(), I can add it if you want.

Go ahead. I'll update my code when you're done.

Quoted 10 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c line 248

Quoted 6 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +3 -0

+	/* Move child to RT partition */
+	SAFE_ASPRINTF(cleanup, &tid_str, "%ld", (long) tid);
+	SAFE_WRITE(cleanup, 1, cpuset_tasks_fd, tid_str,

strlen(tid_str)); +

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +2 -0

+	free(tid_str);
+	SAFE_CLOSE(cleanup, cpuset_tasks_fd);

SAFE_FILE_PRINTF(cleanup, CPUSET_RT_TASKS_FILE, "%ld",
(long)tid) ?

Quoted 2 lines

Or is there a good reason to use open(), asprintf(), write()
and close() instead?

Well, printf() by itself will do no file updates, but I replaced open(), asprintf(), write(), free() and close() with fopen(), fprintf() and fclose().

Quoted 14 lines

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c line 317

Quoted 11 lines

(quoted diff) +1 -0

+	for (nr_matches = fscanf(stream, "%d-%d", &range_first,

&range_last);

(quoted diff) +2 -0

+	     nr_matches > 0;
+	     nr_matches = fscanf(stream, ",%d-%d", &range_first,

&range_last)) {

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c new file +6 -0

+		if (nr_matches == 1)
+			range_last = range_first;
+
+		/* Set all bits in range */
+		for (bit = range_first; bit <= range_last; bit++)
+			mask |= (1 << bit);

Are you sure that this would not overflow?

Quoted 2 lines

I guess that it depends on how you have partitioned your CPUs.

As far as I know there is no API stable and architecture portable way of determining how many CPUs a machine has, which makes this whole business a bit tricky. The problem starts with the partrt tool, which should actually support arbitrarily long masks, but this hasn't happened.

This is on the TODO list, so currently the test only guarantees 32 CPUs. Should probably be documented somewhere...

This test case is intended to test the nohz_full kernel feature. See file testcases/kernel/partrt/README for more information.

utils/Makefile +4 -0

@@ -24,6 +24,10 @@ include $(top_srcdir)/include/mk/env_pre.mk
 
 MAKE_TARGETS         += ffsb
 
+ifneq ($(wildcard rt-tools),)
+MAKE_TARGETS         += partrt list2mask count_ticks
+endif
+
 FFSBDIR			:= ffsb-6.0-rc2
 FILTER_OUT_DIRS		:= $(FFSBDIR)
 FFSB			:= $(FFSBDIR)/ffsb

I could change partrt to accept CPU list directly instead of being dependent on list2mask tool, which means that list2mask disappears.

As for count_ticks, a better name might be kernel_ticks. count_kernel_ticks would be possible but is a bit too long for me...

Would having the tool names

partrt
kernel_ticks

be good enough for ltp, or do you want kernel_ticks prefixed? If so, I'd propose:

partrt
partrt_kernel_ticks

In this case the original name kernel_ticks will be renamed for ltp only.

What do you prefer?

The new version of partrt has a tool named "bitcalc", which performs those nasty bit calculations in a C application instead of an ugly Bash script. This made the partrt dash compatible, and also got rid of list2mask script.

So the new tool names are:
- partrt
- bitcalc
- count_ticks

Having a prefix on bitcalc presents a problem since partrt is using it. This would mean that I have to inform the script that the expected tool is named something else, which I think will make this script a bit ugly.

Prefixing partrt and count_ticks should not be a problem though. Assuming we use partrt as prefix (probably safer than just rt), it would become:

- partrt_partrt
- bitcalc
- partrt_count_ticks

Changes from version 2:
- Added "git submodule init" to README
- Scrapped "verbose" command line parameter entirely, it wasn't implemented anyway
- asprintf() replaced with fprintf() in launch_child()
- Introduced cleanup function parameter to assert_exit_status() and shell(), making cleanup_entered variable in cleanup() obsolete
- Documented 32 CPU limitation in README. The test code itself has been updated to handle all configured CPUs, but the partrt tool does not.
- Simplified tool_available() by using tst_get_path()
- Removed unnecessary exit() and return statements
- Removed rand() to a simple "++" operation as work generator in the RT partition. Seemed like rand() could cause ticks for some reason.
- Use SAFE_POPEN() instead of popen().
- Use a 2 second sleep before counting ticks, as a primitive way of emptying deferred work in the kernel.

Changes from version 1:
- Style fixes proposed by checkpatch.pl script
- Have rt-tools as a git sub-module instead of requiring it to be cloned manually, builds can still be done without checking out rt-tools but you can't run partrt_nohz_full test successfully then
- Check if cpuset supported by kernel, TCONF error if not
- Check if rt-tools has been checked out before build, TCONF error if not
- Simplified determined_nohz_mask()
- Use parse_opts() instead of getopt_long() when parsing command line options
- Moved partrt in SUBDIRS so that it is alphabetically correct in testcases/kernel/Makefile
- launch_child will now handle moving child thread to the right CPUSET partition and CPU, which also eliminates the need for synchronization between parent and child
- Children will no longer be able to cause any tst_* calls
- Removed shell_silent(), above changes made it obsolete
- Removed base parameter to str_to_ulong(), will always be 0 now
- setup_children() is now called from main() rather than test() function.
- Macros defining file names
- Use C code for moving children to the right place instead of external scripts. Using script invocation from RT partition using NFS mounted file system will most likely invoke ticks which can make the test case fail
- Added standard test loop in case reptitions are requested
- Removed atomic operation for cleanup recursion detection, since it is only trying to protect recursive invocations from itself
- Using timerfd to wait for timeout instead of busy-waiting
- Removed pid_popen(), pid_wait() and shell_int()
- assert_exit_status() now also checks if status is -1, i.e. command failed
- Moved _GNU_SOURCE macro from Makefile to C file
- Removed -Werror from Makefile
- Removed err(), fail(), pass() and info() macros
- Replaced usage of READ and WRITE macros with STDIN_FILENO and STDOUT_FILENO, READ and WRITE macros removed
- Removed some unnecessary comments
- setup() is renamed to setup_children()
- Informational message are made mandatory, verbose only affects output from sub-commands, partrt in particular
- Test case name is now partrt_nohz_full

partrt_nohz_full is a test case which checks whether nohz_full mode can achieve 0Hz ticks (if relevant extra patches has been applied) or 1Hz ticks (mainline kernel).

Appropriate kernel configurations and kernel boot parameters needed are documented in testcases/kernel/partrt/README.

Mats Liljegren (1):
partrt_nohz_full: Introducing a new test case

 .gitmodules                                        |    3 +
 README.kernel_config                               |    6 +
 runtest/partrt_nohz_full                           |    1 +
 testcases/kernel/Makefile                          |    1 +
 testcases/kernel/partrt/Makefile                   |   17 +
 testcases/kernel/partrt/README                     |   66 +++
 testcases/kernel/partrt/nohz_full/.gitignore       |    1 +
 testcases/kernel/partrt/nohz_full/Makefile         |   23 +
 .../partrt/nohz_full/test_partrt_nohz_full.c       |  593
++++++++++++++++++++
utils/.gitignore                                   |    3 +
utils/Makefile                                     |    7 +
utils/rt-tools                                     |    1 + 12 files
changed, 722 insertions(+) create mode 100644 runtest/partrt_nohz_full
 create mode 100644 testcases/kernel/partrt/Makefile
 create mode 100644 testcases/kernel/partrt/README
 create mode 100644 testcases/kernel/partrt/nohz_full/.gitignore
 create mode 100644 testcases/kernel/partrt/nohz_full/Makefile
 create mode 100644

testcases/kernel/partrt/nohz_full/test_partrt_nohz_full.c create mode 160000 utils/rt-tools

Ping?