Skip to contentSkip to conversation

x86 perf's dTLB-load-misses broken on IvyBridge?

DISCUSSION Dave Hansen 3 participants 5 messages Feb 18, 2014 – Feb 19, 2014

lore
Share & tools
lore
#0 lore raw

I noticed that perf's dTLB-load-misses even t isn't working on my Ivybridge system:

Quoted 6 lines

Performance counter stats for 'system wide':

                 0      dTLB-load-misses                                             [100.00%]
            48,570      dTLB-store-misses                                            [100.00%]
           202,573      iTLB-loads                                                   [100.00%]
           271,546      iTLB-load-misses          #  134.05% of all iTLB cache hits 

But it works on a SandyBridge system that I have.

arch/x86/kernel/cpu/perf_event_intel.c seems to use the same tables for SandyBridge and IvyBridge, so they both use the 'MEM_UOP_RETIRED.ALL_LOADS' event:

Quoted 5 lines
 [ C(DTLB) ] = {
        [ C(OP_READ) ] = {
                [ C(RESULT_ACCESS) ] = 0x81d0, /* MEM_UOP_RETIRED.ALL_LOADS */
                [ C(RESULT_MISS)   ] = 0x0108, /* DTLB_LOAD_MISSES.CAUSES_A_WALK */
        },

But that event looks to be unsupported on this CPU:

Quoted 7 lines

/ocperf.py stat -a -e mem_uops_retired.all_loads sleep 1
perf stat -a -e cpu/event=0xd0,umask=0x81,name=mem_uops_retired_all_loads/ sleep 1

Performance counter stats for 'system wide':

<not supported> mem_uops_retired_all_loads

        50,204,763      mem_uops_retired_all_loads_ps

But there's a "_ps" version which uses PEBS which does work?

Quoted 2 lines

mem_uops_retired.all_loads [Load uops retired to architected path with filter on bits 0 and 1 applied. (Supports PEBS)]
mem_uops_retired.all_loads_ps [Load uops retired to architected path with filter on bits 0 and 1 applied. (Uses PEBS) (Uses PEBS)]

Should we swap perf_event_intel.c over to use the PEBS version so that it works everywhere?

#1 lore raw
Quoted 16 lines · Dave Hansen

I noticed that perf's dTLB-load-misses even t isn't working on my Ivybridge system:

Quoted 6 lines

Performance counter stats for 'system wide':

                 0      dTLB-load-misses                                             [100.00%]
            48,570      dTLB-store-misses                                            [100.00%]
           202,573      iTLB-loads                                                   [100.00%]
           271,546      iTLB-load-misses          #  134.05% of all iTLB cache hits 

But it works on a SandyBridge system that I have.

arch/x86/kernel/cpu/perf_event_intel.c seems to use the same tables for SandyBridge and IvyBridge, so they both use the 'MEM_UOP_RETIRED.ALL_LOADS' event:

Quoted 9 lines
Quoted 5 lines
 [ C(DTLB) ] = {
        [ C(OP_READ) ] = {
                [ C(RESULT_ACCESS) ] = 0x81d0, /* MEM_UOP_RETIRED.ALL_LOADS */
                [ C(RESULT_MISS)   ] = 0x0108, /* DTLB_LOAD_MISSES.CAUSES_A_WALK */
        },

But that event looks to be unsupported on this CPU:

Quoted 1 line

/ocperf.py stat -a -e mem_uops_retired.all_loads sleep 1

That kind of snake voo-doo is that?

Quoted 1 line
Quoted 1 line

perf stat -a -e cpu/event=0xd0,umask=0x81,name=mem_uops_retired_all_loads/ sleep 1

So this line only produces the mem_uops_retired_all_loads thing, not the _ps thing.

Quoted 7 lines
Quoted 5 lines

Performance counter stats for 'system wide':

<not supported> mem_uops_retired_all_loads

        50,204,763      mem_uops_retired_all_loads_ps

But there's a "_ps" version which uses PEBS which does work?

So clearly there is no _ps version, as I'm still utterly confused as to wtf you mean and where it came from.

Quoted 2 lines
Quoted 2 lines

mem_uops_retired.all_loads [Load uops retired to architected path with filter on bits 0 and 1 applied. (Supports PEBS)]
mem_uops_retired.all_loads_ps [Load uops retired to architected path with filter on bits 0 and 1 applied. (Uses PEBS) (Uses PEBS)]

What's that; SDM not haz this.

Quoted 2 lines

Should we swap perf_event_intel.c over to use the PEBS version so that it works everywhere?

I'm confused; where does this _ps thing come from? There's nothing like that in the SDM. That only lists the D0H event, and says it should work.

Of course the SDM is trying to confuse the living daylight out of people by calling it crap like "3rd gen intel core", which just shows they can't bloody well count either, since it went:

core, core2, nhm, wsm, snb, ivb

do its damn well 6th gen.

#2 lore raw
In reply to Dave Hansen
Quoted 16 lines · Dave Hansen

I noticed that perf's dTLB-load-misses even t isn't working on my Ivybridge system:

Quoted 6 lines

Performance counter stats for 'system wide':

                 0      dTLB-load-misses                                             [100.00%]
            48,570      dTLB-store-misses                                            [100.00%]
           202,573      iTLB-loads                                                   [100.00%]
           271,546      iTLB-load-misses          #  134.05% of all iTLB cache hits 

But it works on a SandyBridge system that I have.

arch/x86/kernel/cpu/perf_event_intel.c seems to use the same tables for SandyBridge and IvyBridge, so they both use the 'MEM_UOP_RETIRED.ALL_LOADS' event:

Quoted 7 lines
Quoted 5 lines
 [ C(DTLB) ] = {
        [ C(OP_READ) ] = {
                [ C(RESULT_ACCESS) ] = 0x81d0, /* MEM_UOP_RETIRED.ALL_LOADS */
                [ C(RESULT_MISS)   ] = 0x0108, /* DTLB_LOAD_MISSES.CAUSES_A_WALK */
        },

But that event looks to be unsupported on this CPU:

I thought you wanted the miss event?

That would be the second entry.

ALL_LOADS is the access event. it works for me, both raw and perf cooked (not sure why the two numbers are different though)

% perf stat -e dTLB-loads,r81d0 -a sleep 1

Performance counter stats for 'system wide':

        12,685,064 dTLB-loads                                                   [100.00%]
        13,277,367 r81d0                                                       

       1.001420860 seconds time elapsed

Miss event count too:

perf stat -e dTLB-load-misses,dTLB-load -a sleep 1

Performance counter stats for 'system wide':

            19,504 dTLB-load-misses          #    0.30% of all dTLB cache hits  [100.00%]
         6,471,308 dTLB-load                                                   

       1.001894328 seconds time elapsed

Same raw:

perf stat -e r0108 -a sleep 1

Performance counter stats for 'system wide':

            82,285 r0108                                                       

       1.001353060 seconds time elapsed
Quoted 8 lines
Quoted 6 lines

perf stat -a -e cpu/event=0xd0,umask=0x81,name=mem_uops_retired_all_loads/ sleep 1

Performance counter stats for 'system wide':

<not supported> mem_uops_retired_all_loads

        50,204,763      mem_uops_retired_all_loads_ps

But there's a "_ps" version which uses PEBS which does work?

Both works for me on a IvyBridge.

Quoted 2 lines

Should we swap perf_event_intel.c over to use the PEBS version so that it works everywhere?

Shouldn't be needed.

PEBS for counting normally doesn't make much sense.

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
#3 lore raw

Re: x86 perf's dTLB-load-misses broken on IvyBridge? II

In reply to Dave Hansen
Quoted 5 lines
Quoted 5 lines
 [ C(DTLB) ] = {
        [ C(OP_READ) ] = {
                [ C(RESULT_ACCESS) ] = 0x81d0, /* MEM_UOP_RETIRED.ALL_LOADS */
                [ C(RESULT_MISS)   ] = 0x0108, /* DTLB_LOAD_MISSES.CAUSES_A_WALK */
        },

Actually I tested on the wrong system earlier, sorry The miss event code
is wrong on IvyBridge, it should be 0x8108 (DTLB_LOAD_MISSES.DEMAND_LD_MISS_CAUSES_A_WALK), not 0x0108.

BTW that event only counts demand loads, no prefetches, that would be a separate event.

dTLB-loads works with and without PEBS.

-Andi

#4 lore raw

Re: x86 perf's dTLB-load-misses broken on IvyBridge? II

Quoted 10 lines · Andi Kleen
Quoted 5 lines
Quoted 5 lines
 [ C(DTLB) ] = {
        [ C(OP_READ) ] = {
                [ C(RESULT_ACCESS) ] = 0x81d0, /* MEM_UOP_RETIRED.ALL_LOADS */
                [ C(RESULT_MISS)   ] = 0x0108, /* DTLB_LOAD_MISSES.CAUSES_A_WALK */
        },

Actually I tested on the wrong system earlier, sorry The miss event code
is wrong on IvyBridge, it should be 0x8108 (DTLB_LOAD_MISSES.DEMAND_LD_MISS_CAUSES_A_WALK), not 0x0108.

Ah indeed, that is different between snb and ivb.