Skip to contentSkip to conversation

perf-core build fails on powerpc

DISCUSSION Sukadev Bhattiprolu 3 participants 8 messages Mar 13, 2018 – Mar 14, 2018 perf events

lore
Share & tools
lore
#0 lore raw

Building perf on Powerpc seems broken when using Arnaldo's perf/core branch with HEAD as:

1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")

It maybe related to this commit:

	commit d596299
	Author: John Garry <[email protected]>
	Date:   Thu Mar 8 18:58:29 2018 +0800

	    perf vendor events: Add support for pmu events vendor subdirectory

Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the problem for me.

@@ -957,8 +963,12 @@ static int process_one_file(const char *fpath, const struct

        /* model directory, reset topic */
+#if 0
        if ((level == 1 && is_dir && is_leaf_dir(fpath)) ||
            (level == 2 && is_dir)) {

+#else
+ if (level == 1 && is_dir) {
+#endif

The problem is that the tools/perf/pmu-events/pmu-events.c file generated by the jevents program is missing the "prefix" lines like:

	struct pmu_event pme_power8[] = {
and 
	struct pmu_event pme_power9[] = {

i.e the pmu-events.c begins like:
---

#include "../../pmu-events/pmu-events.h"

{
        .name = "pm_data_from_dl2l3_mod",
        .event = "event=0x4c048",
        .desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to a demand load",
        .topic = "cache",
        .long_desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to either only demand loads or demand loads plus prefetches if MMCR1[16] is 1",
},

---

Below is the debug and info messages at the time of failure:

---

  HOSTCC   /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents.o
  HOSTLD   /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents-in.o
  LINK     /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents
  GEN      /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c
d 0      50 powerpc              pmu-events/arch/powerpc
d 1    4096 power8               pmu-events/arch/powerpc/power8
f 2   10275 cache.json           pmu-events/arch/powerpc/power8/cache.json
f 2     302 floating-point.json  pmu-events/arch/powerpc/power8/floating-point.json
f 2   29283 marked.json          pmu-events/arch/powerpc/power8/marked.json
f 2   12028 memory.json          pmu-events/arch/powerpc/power8/memory.json
f 2  159271 other.json           pmu-events/arch/powerpc/power8/other.json
f 2   10777 pipeline.json        pmu-events/arch/powerpc/power8/pipeline.json
f 2    3781 pmc.json             pmu-events/arch/powerpc/power8/pmc.json
f 2    6664 translation.json     pmu-events/arch/powerpc/power8/translation.json
f 2   25104 frontend.json        pmu-events/arch/powerpc/power8/frontend.json
d 1    4096 power9               pmu-events/arch/powerpc/power9
f 2    1096 floating-point.json  pmu-events/arch/powerpc/power9/floating-point.json
f 2    5503 cache.json           pmu-events/arch/powerpc/power9/cache.json
f 2   14885 frontend.json        pmu-events/arch/powerpc/power9/frontend.json
f 2   27807 marked.json          pmu-events/arch/powerpc/power9/marked.json
f 2    4820 memory.json          pmu-events/arch/powerpc/power9/memory.json
f 2   85387 other.json           pmu-events/arch/powerpc/power9/other.json
f 2   23839 pipeline.json        pmu-events/arch/powerpc/power9/pipeline.json
f 2    5138 pmc.json             pmu-events/arch/powerpc/power9/pmc.json
f 2    9897 translation.json     pmu-events/arch/powerpc/power9/translation.json
f 1     440 mapfile.csv          pmu-events/arch/powerpc/mapfile.csv

jevents: Processing mapfile pmu-events/arch/powerpc/mapfile.csv
CC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.o
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:2:1: error: expected identifier or ‘(’ before ‘{’ token
{
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:8:2: error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:15:2: error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:22:2: error: expected identifier or ‘(’ before ‘,’ token
},

----

#1 lore raw
Quoted 50 lines · Sukadev Bhattiprolu

Building perf on Powerpc seems broken when using Arnaldo's perf/core branch with HEAD as:

1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")

It maybe related to this commit:

	commit d596299
	Author: John Garry <[email protected]>
	Date:   Thu Mar 8 18:58:29 2018 +0800

	    perf vendor events: Add support for pmu events vendor subdirectory

Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the problem for me.

(quoted diff) +4 -0
@@ -957,8 +963,12 @@ static int process_one_file(const char *fpath, const struct
 
 
        /* model directory, reset topic */
+#if 0
        if ((level == 1 && is_dir && is_leaf_dir(fpath)) ||
            (level == 2 && is_dir)) {
+#else
+       if (level == 1 && is_dir) {
+#endif
 
 

The problem is that the tools/perf/pmu-events/pmu-events.c file generated by the jevents program is missing the "prefix" lines like:

	struct pmu_event pme_power8[] = {
and
	struct pmu_event pme_power9[] = {

i.e the pmu-events.c begins like:
---

#include "../../pmu-events/pmu-events.h"

{
        .name = "pm_data_from_dl2l3_mod",
        .event = "event=0x4c048",
        .desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to a demand load",
        .topic = "cache",
        .long_desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to either only demand loads or demand loads plus prefetches if MMCR1[16] is 1",
},
Quoted 2 lines

---

I will check this, thanks and sorry for the hassle.

John

Quoted 49 lines

Below is the debug and info messages at the time of failure:

---

  HOSTCC   /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents.o
  HOSTLD   /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents-in.o
  LINK     /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents
  GEN      /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c
d 0      50 powerpc              pmu-events/arch/powerpc
d 1    4096 power8               pmu-events/arch/powerpc/power8
f 2   10275 cache.json           pmu-events/arch/powerpc/power8/cache.json
f 2     302 floating-point.json  pmu-events/arch/powerpc/power8/floating-point.json
f 2   29283 marked.json          pmu-events/arch/powerpc/power8/marked.json
f 2   12028 memory.json          pmu-events/arch/powerpc/power8/memory.json
f 2  159271 other.json           pmu-events/arch/powerpc/power8/other.json
f 2   10777 pipeline.json        pmu-events/arch/powerpc/power8/pipeline.json
f 2    3781 pmc.json             pmu-events/arch/powerpc/power8/pmc.json
f 2    6664 translation.json     pmu-events/arch/powerpc/power8/translation.json
f 2   25104 frontend.json        pmu-events/arch/powerpc/power8/frontend.json
d 1    4096 power9               pmu-events/arch/powerpc/power9
f 2    1096 floating-point.json  pmu-events/arch/powerpc/power9/floating-point.json
f 2    5503 cache.json           pmu-events/arch/powerpc/power9/cache.json
f 2   14885 frontend.json        pmu-events/arch/powerpc/power9/frontend.json
f 2   27807 marked.json          pmu-events/arch/powerpc/power9/marked.json
f 2    4820 memory.json          pmu-events/arch/powerpc/power9/memory.json
f 2   85387 other.json           pmu-events/arch/powerpc/power9/other.json
f 2   23839 pipeline.json        pmu-events/arch/powerpc/power9/pipeline.json
f 2    5138 pmc.json             pmu-events/arch/powerpc/power9/pmc.json
f 2    9897 translation.json     pmu-events/arch/powerpc/power9/translation.json
f 1     440 mapfile.csv          pmu-events/arch/powerpc/mapfile.csv

jevents: Processing mapfile pmu-events/arch/powerpc/mapfile.csv
CC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.o
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:2:1: error: expected identifier or ‘(’ before ‘{’ token
{
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:8:2: error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:15:2: error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:22:2: error: expected identifier or ‘(’ before ‘,’ token
},

----

.

#2 lore raw
Quoted 14 lines · John Garry [[email protected]] wrote:
Quoted 13 lines · Sukadev Bhattiprolu

Building perf on Powerpc seems broken when using Arnaldo's perf/core branch with HEAD as:

1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")

It maybe related to this commit:

	commit d596299
	Author: John Garry <[email protected]>
	Date:   Thu Mar 8 18:58:29 2018 +0800
Quoted 5 lines
Quoted 5 lines

perf vendor events: Add support for pmu events vendor subdirectory

Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the problem for me.

Hi John,

I have an xfs file system which seems to have d_type == DT_UNKNOWN for all entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type may not be supported by all file systems.

Not relying on ->d_type seems to fix it:

@@ -873,26 +879,26 @@ static int is_leaf_dir(const char *fpath)

                return 0;
 
        while ((dir = readdir(d)) != NULL) {

- if (dir->d_type == DT_DIR && dir->d_name[0] != '.') {
- res = 0;
- break;
- } else if (dir->d_type == DT_UNKNOWN) {
- char path[PATH_MAX];
- struct stat st;
+ char path[PATH_MAX];
+ struct stat st;

- sprintf(path, "%s/%s", fpath, dir->d_name);
- if (stat(path, &st))
- break;
+ if (strcmp(dir->d_name, ".") == 0 ||
+ strcmp(dir->d_name, "..") == 0)
+ continue;

- if (S_ISDIR(st.st_mode)) {
- res = 0;
- break;
- }
+ sprintf(path, "%s/%s", fpath, dir->d_name);
+ if (stat(path, &st))
+ break;
+
+ if (S_ISDIR(st.st_mode)) {
+ res = 0;
+ break;

#3 lore raw

On 13/03/2018 20:10, Sukadev Bhattiprolu wrote:

+

Quoted 21 lines
Quoted 14 lines · John Garry [[email protected]] wrote:
Quoted 13 lines · Sukadev Bhattiprolu

Building perf on Powerpc seems broken when using Arnaldo's perf/core branch with HEAD as:

1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")

It maybe related to this commit:

	commit d596299
	Author: John Garry <[email protected]>
	Date:   Thu Mar 8 18:58:29 2018 +0800
Quoted 5 lines
Quoted 5 lines

perf vendor events: Add support for pmu events vendor subdirectory

Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the problem for me.

Quoted 8 lines

Hi John,

I have an xfs file system which seems to have d_type == DT_UNKNOWN for all entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type may not be supported by all file systems.

Not relying on ->d_type seems to fix it:

Hi Sukadev,

Thanks for debugging this. Jiri Olsa (cc'ed) warned me on this, so I did add the check for d_type == DT_UNKNOWN.

But, if all files have d_type == DT_UNKNOWN, you're code would from visual observation look to be same as mine (apart from check for '.' or '..' filename, which I would say is already covered by stat() and S_ISDIR()). Or is d_type value just unreliable?

Much appreciated,
John

Quoted 34 lines
(quoted diff) +12 -13
@@ -873,26 +879,26 @@ static int is_leaf_dir(const char *fpath)
                return 0;
 
        while ((dir = readdir(d)) != NULL) {
-               if (dir->d_type == DT_DIR && dir->d_name[0] != '.') {
-                       res = 0;
-                       break;
-               } else if (dir->d_type == DT_UNKNOWN) {
-                       char path[PATH_MAX];
-                       struct stat st;
+               char path[PATH_MAX];
+               struct stat st;
 
-                       sprintf(path, "%s/%s", fpath, dir->d_name);
-                       if (stat(path, &st))
-                               break;
+               if (strcmp(dir->d_name, ".") == 0 ||
+                               strcmp(dir->d_name, "..") == 0)
+                       continue;
 
-                       if (S_ISDIR(st.st_mode)) {
-                               res = 0;
-                               break;
-                       }
+               sprintf(path, "%s/%s", fpath, dir->d_name);
+               if (stat(path, &st))
+                       break;
+
+               if (S_ISDIR(st.st_mode)) {
+                       res = 0;
+                       break;
 
 
 
#4 lore raw
Quoted 13 lines · John Garry [[email protected]] wrote:
Quoted 8 lines · Sukadev Bhattiprolu

Hi John,

I have an xfs file system which seems to have d_type == DT_UNKNOWN for all entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type may not be supported by all file systems.

Not relying on ->d_type seems to fix it:

Hi Sukadev,

Quoted 7 lines

Thanks for debugging this. Jiri Olsa (cc'ed) warned me on this, so I did add the check for d_type == DT_UNKNOWN.

But, if all files have d_type == DT_UNKNOWN, you're code would from visual observation look to be same as mine (apart from check for '.' or '..' filename, which I would say is already covered by stat() and S_ISDIR()). Or is d_type value just unreliable?

In the current code and with DT_UNKNOWN, is_leaf_dir() returns false when it sees the "." or ".." entries right? In the new code, we skip those and return false only if we find some other directory.

Thanks,

Sukadev

#7 lore raw
In reply to Sukadev Bhattiprolu

Em Tue, Mar 13, 2018 at 12:17:42PM -0700, Sukadev Bhattiprolu escreveu:

Quoted 8 lines

Building perf on Powerpc seems broken when using Arnaldo's perf/core branch with HEAD as:

1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")

It maybe related to this commit:

Humpf, this should've been caught by one of the powerpc crossbuild containers I have in place... will check why the jevents.c code is not being stressed...

- Arnaldo

Quoted 88 lines
	commit d596299
	Author: John Garry <[email protected]>
	Date:   Thu Mar 8 18:58:29 2018 +0800

	    perf vendor events: Add support for pmu events vendor subdirectory

Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the problem for me.

(quoted diff) +4 -0
@@ -957,8 +963,12 @@ static int process_one_file(const char *fpath, const struct
 
 
        /* model directory, reset topic */
+#if 0
        if ((level == 1 && is_dir && is_leaf_dir(fpath)) ||
            (level == 2 && is_dir)) {
+#else
+       if (level == 1 && is_dir) {
+#endif
 
 

The problem is that the tools/perf/pmu-events/pmu-events.c file generated by the jevents program is missing the "prefix" lines like:

	struct pmu_event pme_power8[] = {
and 
	struct pmu_event pme_power9[] = {

i.e the pmu-events.c begins like:
---

#include "../../pmu-events/pmu-events.h"

{
        .name = "pm_data_from_dl2l3_mod",
        .event = "event=0x4c048",
        .desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to a demand load",
        .topic = "cache",
        .long_desc = "The processor's data cache was reloaded with Modified (M) data from another chip's L2 or L3 on a different Node or Group (Distant), as this chip due to either only demand loads or demand loads plus prefetches if MMCR1[16] is 1",
},

---

Below is the debug and info messages at the time of failure:

---

  HOSTCC   /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents.o
  HOSTLD   /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents-in.o
  LINK     /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents
  GEN      /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c
d 0      50 powerpc              pmu-events/arch/powerpc
d 1    4096 power8               pmu-events/arch/powerpc/power8
f 2   10275 cache.json           pmu-events/arch/powerpc/power8/cache.json
f 2     302 floating-point.json  pmu-events/arch/powerpc/power8/floating-point.json
f 2   29283 marked.json          pmu-events/arch/powerpc/power8/marked.json
f 2   12028 memory.json          pmu-events/arch/powerpc/power8/memory.json
f 2  159271 other.json           pmu-events/arch/powerpc/power8/other.json
f 2   10777 pipeline.json        pmu-events/arch/powerpc/power8/pipeline.json
f 2    3781 pmc.json             pmu-events/arch/powerpc/power8/pmc.json
f 2    6664 translation.json     pmu-events/arch/powerpc/power8/translation.json
f 2   25104 frontend.json        pmu-events/arch/powerpc/power8/frontend.json
d 1    4096 power9               pmu-events/arch/powerpc/power9
f 2    1096 floating-point.json  pmu-events/arch/powerpc/power9/floating-point.json
f 2    5503 cache.json           pmu-events/arch/powerpc/power9/cache.json
f 2   14885 frontend.json        pmu-events/arch/powerpc/power9/frontend.json
f 2   27807 marked.json          pmu-events/arch/powerpc/power9/marked.json
f 2    4820 memory.json          pmu-events/arch/powerpc/power9/memory.json
f 2   85387 other.json           pmu-events/arch/powerpc/power9/other.json
f 2   23839 pipeline.json        pmu-events/arch/powerpc/power9/pipeline.json
f 2    5138 pmc.json             pmu-events/arch/powerpc/power9/pmc.json
f 2    9897 translation.json     pmu-events/arch/powerpc/power9/translation.json
f 1     440 mapfile.csv          pmu-events/arch/powerpc/mapfile.csv

jevents: Processing mapfile pmu-events/arch/powerpc/mapfile.csv
CC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.o
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:2:1: error: expected identifier or ‘(’ before ‘{’ token
{
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:8:2: error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:15:2: error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:22:2: error: expected identifier or ‘(’ before ‘,’ token
},

----

#5 lore raw
In reply to Sukadev Bhattiprolu
Quoted 17 lines · Sukadev Bhattiprolu
Quoted 16 lines · John Garry [[email protected]] wrote:
Quoted 8 lines · Sukadev Bhattiprolu

Hi John,

I have an xfs file system which seems to have d_type == DT_UNKNOWN for all entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type may not be supported by all file systems.

Not relying on ->d_type seems to fix it:

Hi Sukadev,

Thanks for debugging this. Jiri Olsa (cc'ed) warned me on this, so I did add the check for d_type == DT_UNKNOWN.

Quoted 9 lines
Quoted 4 lines

But, if all files have d_type == DT_UNKNOWN, you're code would from visual observation look to be same as mine (apart from check for '.' or '..' filename, which I would say is already covered by stat() and S_ISDIR()). Or is d_type value just unreliable?

In the current code and with DT_UNKNOWN, is_leaf_dir() returns false when it sees the "." or ".." entries right? In the new code, we skip those and return false only if we find some other directory.

OK, that's the real issue, being the "." and ".." entries.

@Arnaldo, I'll try to send a fix for this today. I just need to make an xfs to test. Sorry for the hassle.

John

Quoted 6 lines

Thanks,

Sukadev

#6 lore raw
Quoted 33 lines · John Garry
Quoted 27 lines · Sukadev Bhattiprolu
Quoted 19 lines · John Garry [[email protected]] wrote:
Quoted 10 lines · Sukadev Bhattiprolu

Hi John,

I have an xfs file system which seems to have d_type == DT_UNKNOWN for all
entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type
may not be supported by all file systems.

Not relying on ->d_type seems to fix it:

Hi Sukadev,

Thanks for debugging this. Jiri Olsa (cc'ed) warned me on this, so I did add
the check for d_type == DT_UNKNOWN.

Quoted 6 lines

But, if all files have d_type == DT_UNKNOWN, you're code would from visual
observation look to be same as mine (apart from check for '.' or '..' filename, which I would say is already covered by stat() and S_ISDIR()). Or
is d_type value just unreliable?

Quoted 4 lines

In the current code and with DT_UNKNOWN, is_leaf_dir() returns false when it sees the "." or ".." entries right? In the new code, we skip those and return false only if we find some other directory.

Quoted 5 lines

OK, that's the real issue, being the "." and ".." entries.

@Arnaldo, I'll try to send a fix for this today. I just need to make an xfs to test. Sorry for the hassle.

Hi Sukadev,

Even with the xfs I have created I still don't see DT_UNKNOWN. So can you kindly test this change for me:

tools/perf/pmu-events/jevents.c +4 -1
index 1c018445e757..9f712dafa0b2 100644
@@ -873,7 +873,10 @@ static int is_leaf_dir(const char *fpath)
                return 0;
 
        while ((dir = readdir(d)) != NULL) {
-               if (dir->d_type == DT_DIR && dir->d_name[0] != '.') {
+               if (!strcmp(dir->d_name, ".") || !strcmp(dir->d_name, ".."))
+                    continue;
+
+               if (dir->d_type == DT_DIR) {
                        res = 0;
                        break;
                } else if (dir->d_type == DT_UNKNOWN) {

It's quite similar to your change, except keeping the check for DT_DIR.

Much appreciated,
John

Quoted 9 lines

John

Quoted 6 lines

Thanks,

Sukadev