Skip to contentSkip to conversation

perf test: Fix dso cache testcase

PATCH In review Namhyung Kim 3 participants 8 messages Jan 30, 2015 – Feb 18, 2015 perf events

lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

PATCH 1/3 perf test: Fix dso cache testcase

The current dso cache permits to keep dso->data.fd is open under a half of open file limit. But test__dso_data_cache() sets dso_cnt to limit / 2 + 1 so it'll reach the limit in the loop even though the loop count is one less than the dso_cnt and it makes the final dso__data_fd() after the loop meaningless.

I guess the intention was dsos[0]->data.fd is open before the last open and gets closed after it. So add an assert before the last open.

---
 tools/perf/tests/dso-data.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/perf/tests/dso-data.c +6 -3
index caaf37f079b1..22a8c428283a 100644
@@ -243,8 +243,8 @@ int test__dso_data_cache(void)
 	limit = nr * 4;
 	TEST_ASSERT_VAL("failed to set file limit", !set_fd_limit(limit));
 
-	/* and this is now our dso open FDs limit + 1 extra */
-	dso_cnt = limit / 2 + 1;
+	/* and this is now our dso open FDs limit */
+	dso_cnt = limit / 2;
 	TEST_ASSERT_VAL("failed to create dsos\n",
 		!dsos__create(dso_cnt, TEST_FILE_SIZE));
 
@@ -268,7 +268,10 @@ int test__dso_data_cache(void)
 		}
 	}
 
-	/* open +1 dso over the allowed limit */
+	/* verify the first one is already open */
+	TEST_ASSERT_VAL("dsos[0] is not open", dsos[0]->data.fd != -1);
+
+	/* open +1 dso to reach the allowed limit */
 	fd = dso__data_fd(dsos[i], &machine);
 	TEST_ASSERT_VAL("failed to get fd", fd > 0);
 
Signature · 1 line
2.2.2
author maintainer
#1 lore raw

PATCH 2/3 perf tools: Do not rely on dso__data_read_offset() to open dso

Do not rely on dso__data_read_offset() will always call dso__data_fd() internally. With multi-thread support, accessing a fd will be protected by a lock and it'll cause a huge contention. It can be avoided since we can skip reading from file if there's a data in the dso cache.

If one needs to call the dso__data_read_offset(), [s]he also needs to call dso__data_fd() (or set dso->binary_type at least) first like the dwarf unwind code does.

---
 tools/perf/tests/dso-data.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
tools/perf/tests/dso-data.c +8 -5
index 22a8c428283a..513e5febbe5a 100644
@@ -112,6 +112,9 @@ int test__dso_data(void)
 
 	dso = dso__new((const char *)file);
 
+	TEST_ASSERT_VAL("Failed to access to dso",
+			dso__data_fd(dso, &machine) >= 0);
+
 	/* Basic 10 bytes tests. */
 	for (i = 0; i < ARRAY_SIZE(offsets); i++) {
 		struct test_data_offset *data = &offsets[i];
@@ -252,13 +255,13 @@ int test__dso_data_cache(void)
 		struct dso *dso = dsos[i];
 
 		/*
-		 * Open dsos via dso__data_fd or dso__data_read_offset.
-		 * Both opens the data file and keep it open.
+		 * Open dsos via dso__data_fd(), it opens the data
+		 * file and keep it open (unless open file limit).
 		 */
+		fd = dso__data_fd(dso, &machine);
+		TEST_ASSERT_VAL("failed to get fd", fd > 0);
+
 		if (i % 2) {
-			fd = dso__data_fd(dso, &machine);
-			TEST_ASSERT_VAL("failed to get fd", fd > 0);
-		} else {
 			#define BUFSIZE 10
 			u8 buf[BUFSIZE];
 			ssize_t n;
Signature · 1 line
2.2.2
Acked-by on v1 2/3
#2 lore raw

Re: [PATCH 2/3] perf tools: Do not rely on dso__data_read_offset() to open dso

Quoted 9 lines · Namhyung Kim

Do not rely on dso__data_read_offset() will always call dso__data_fd() internally. With multi-thread support, accessing a fd will be protected by a lock and it'll cause a huge contention. It can be avoided since we can skip reading from file if there's a data in the dso cache.

If one needs to call the dso__data_read_offset(), [s]he also needs to call dso__data_fd() (or set dso->binary_type at least) first like the dwarf unwind code does.

hum, I dont see the need for this, but I haven't checked your patchset for the multithread support yet ;-)

anyway, I dont mind tp straighten up the interface:

thanks,
jirka

#3 lore raw

[tip:perf/core] perf tests: Do not rely on dso__data_read_offset( ) to open dso

In reply to Namhyung Kim
perf tests: Do not rely on dso__data_read_offset() to open dso
Commit-ID:  63d3c6f3835d011c783c606c8a1583b041f579aa
Gitweb:     http://git.kernel.org/tip/63d3c6f3835d011c783c606c8a1583b041f579aa
Author:     Namhyung Kim <[email protected]>
AuthorDate: Fri, 30 Jan 2015 11:33:28 +0900
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 6 Feb 2015 11:46:35 +0100

perf tests: Do not rely on dso__data_read_offset() to open dso

Do not rely on dso__data_read_offset() will always call dso__data_fd() internally. With multi-thread support, accessing a fd will be protected by a lock and it'll cause a huge contention. It can be avoided since we can skip reading from file if there's a data in the dso cache.

If one needs to call the dso__data_read_offset(), [s]he also needs to call dso__data_fd() (or set dso->binary_type at least) first like the dwarf unwind code does.

---
 tools/perf/tests/dso-data.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
tools/perf/tests/dso-data.c +8 -5
index 22a8c42..513e5fe 100644
@@ -112,6 +112,9 @@ int test__dso_data(void)
 
 	dso = dso__new((const char *)file);
 
+	TEST_ASSERT_VAL("Failed to access to dso",
+			dso__data_fd(dso, &machine) >= 0);
+
 	/* Basic 10 bytes tests. */
 	for (i = 0; i < ARRAY_SIZE(offsets); i++) {
 		struct test_data_offset *data = &offsets[i];
@@ -252,13 +255,13 @@ int test__dso_data_cache(void)
 		struct dso *dso = dsos[i];
 
 		/*
-		 * Open dsos via dso__data_fd or dso__data_read_offset.
-		 * Both opens the data file and keep it open.
+		 * Open dsos via dso__data_fd(), it opens the data
+		 * file and keep it open (unless open file limit).
 		 */
+		fd = dso__data_fd(dso, &machine);
+		TEST_ASSERT_VAL("failed to get fd", fd > 0);
+
 		if (i % 2) {
-			fd = dso__data_fd(dso, &machine);
-			TEST_ASSERT_VAL("failed to get fd", fd > 0);
-		} else {
 			#define BUFSIZE 10
 			u8 buf[BUFSIZE];
 			ssize_t n;
author maintainer
#4 lore raw

PATCH 3/3 perf tools: Fix a dso open fail message

In reply to Namhyung Kim

It's not related to mmap, remove it from the message.

---
 tools/perf/util/dso.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/dso.c +1 -1
index c2f7d3b90966..a8b3f18db1a5 100644
@@ -240,7 +240,7 @@ static int do_open(char *name)
 		if (fd >= 0)
 			return fd;
 
-		pr_debug("dso open failed, mmap: %s\n",
+		pr_debug("dso open failed: %s\n",
 			 strerror_r(errno, sbuf, sizeof(sbuf)));
 		if (!dso__data_open_cnt || errno != EMFILE)
 			break;
Signature · 1 line
2.2.2
#5 lore raw

[tip:perf/core] perf tools: Fix a dso open fail message

perf tools: Fix a dso open fail message
Commit-ID:  a3c0cc2ac03bd9db032f590d59cdbf0b447503b8
Gitweb:     http://git.kernel.org/tip/a3c0cc2ac03bd9db032f590d59cdbf0b447503b8
Author:     Namhyung Kim <[email protected]>
AuthorDate: Fri, 30 Jan 2015 11:33:29 +0900
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 6 Feb 2015 11:46:36 +0100

perf tools: Fix a dso open fail message

It's not related to mmap, remove it from the message.

---
 tools/perf/util/dso.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/dso.c +1 -1
index c2f7d3b..a8b3f18 100644
@@ -240,7 +240,7 @@ static int do_open(char *name)
 		if (fd >= 0)
 			return fd;
 
-		pr_debug("dso open failed, mmap: %s\n",
+		pr_debug("dso open failed: %s\n",
 			 strerror_r(errno, sbuf, sizeof(sbuf)));
 		if (!dso__data_open_cnt || errno != EMFILE)
 			break;
Acked-by on v1 1/3
#6 lore raw
In reply to Namhyung Kim
Quoted 10 lines · Namhyung Kim

The current dso cache permits to keep dso->data.fd is open under a half of open file limit. But test__dso_data_cache() sets dso_cnt to limit / 2 + 1 so it'll reach the limit in the loop even though the loop count is one less than the dso_cnt and it makes the final dso__data_fd() after the loop meaningless.

I guess the intention was dsos[0]->data.fd is open before the last open and gets closed after it. So add an assert before the last open.

thanks,
jirka

Quoted 34 lines
---
 tools/perf/tests/dso-data.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/perf/tests/dso-data.c +6 -3
index caaf37f079b1..22a8c428283a 100644
@@ -243,8 +243,8 @@ int test__dso_data_cache(void)
 	limit = nr * 4;
 	TEST_ASSERT_VAL("failed to set file limit", !set_fd_limit(limit));
 
-	/* and this is now our dso open FDs limit + 1 extra */
-	dso_cnt = limit / 2 + 1;
+	/* and this is now our dso open FDs limit */
+	dso_cnt = limit / 2;
 	TEST_ASSERT_VAL("failed to create dsos\n",
 		!dsos__create(dso_cnt, TEST_FILE_SIZE));
 
@@ -268,7 +268,10 @@ int test__dso_data_cache(void)
 		}
 	}
 
-	/* open +1 dso over the allowed limit */
+	/* verify the first one is already open */
+	TEST_ASSERT_VAL("dsos[0] is not open", dsos[0]->data.fd != -1);
+
+	/* open +1 dso to reach the allowed limit */
 	fd = dso__data_fd(dsos[i], &machine);
 	TEST_ASSERT_VAL("failed to get fd", fd > 0);
 
Signature · 1 line
2.2.2
#7 lore raw
In reply to Namhyung Kim
perf test: Fix dso cache testcase
Commit-ID:  66af43d56345a7ca549ba1089fe11a6953072417
Gitweb:     http://git.kernel.org/tip/66af43d56345a7ca549ba1089fe11a6953072417
Author:     Namhyung Kim <[email protected]>
AuthorDate: Fri, 30 Jan 2015 11:33:27 +0900
Committer:  Arnaldo Carvalho de Melo <[email protected]>
CommitDate: Fri, 6 Feb 2015 11:46:35 +0100

perf test: Fix dso cache testcase

The current dso cache permits to keep dso->data.fd is open under a half of open file limit. But test__dso_data_cache() sets dso_cnt to limit / 2 + 1 so it'll reach the limit in the loop even though the loop count is one less than the dso_cnt and it makes the final dso__data_fd() after the loop meaningless.

I guess the intention was dsos[0]->data.fd is open before the last open and gets closed after it. So add an assert before the last open.

---
 tools/perf/tests/dso-data.c | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
tools/perf/tests/dso-data.c +6 -3
index caaf37f..22a8c42 100644
@@ -243,8 +243,8 @@ int test__dso_data_cache(void)
 	limit = nr * 4;
 	TEST_ASSERT_VAL("failed to set file limit", !set_fd_limit(limit));
 
-	/* and this is now our dso open FDs limit + 1 extra */
-	dso_cnt = limit / 2 + 1;
+	/* and this is now our dso open FDs limit */
+	dso_cnt = limit / 2;
 	TEST_ASSERT_VAL("failed to create dsos\n",
 		!dsos__create(dso_cnt, TEST_FILE_SIZE));
 
@@ -268,7 +268,10 @@ int test__dso_data_cache(void)
 		}
 	}
 
-	/* open +1 dso over the allowed limit */
+	/* verify the first one is already open */
+	TEST_ASSERT_VAL("dsos[0] is not open", dsos[0]->data.fd != -1);
+
+	/* open +1 dso to reach the allowed limit */
 	fd = dso__data_fd(dsos[i], &machine);
 	TEST_ASSERT_VAL("failed to get fd", fd > 0);