PEBS virtualization enabling via DS

From: Kan Liang <[email protected]>

The PEBS event created by host needs to be assigned specific counters requested by the guest, which means the guest and host counter indexes have to be the same or fail to create. This is needed because PEBS leaks counter indexes into the guest. Otherwise, the guest driver will be confused by the counter indexes in the status field of the PEBS record.

A guest_dedicated_idx field is added to indicate the counter index specifically requested by KVM. The dedicated event constraints would constrain the counter in the host to the same numbered counter in guest.

A intel_ctrl_guest_dedicated_mask field is added to indicate the enabled counters for guest PEBS events. The IA32_PEBS_ENABLE MSR will be switched during the VMX transitions if intel_ctrl_guest_owned is set.

Quoted 16 lines · Luwei Kang

From: Kan Liang <[email protected]>

The PEBS event created by host needs to be assigned specific counters requested by the guest, which means the guest and host counter indexes have to be the same or fail to create. This is needed because PEBS leaks counter indexes into the guest. Otherwise, the guest driver will be confused by the counter indexes in the status field of the PEBS record.

A guest_dedicated_idx field is added to indicate the counter index specifically requested by KVM. The dedicated event constraints would constrain the counter in the host to the same numbered counter in guest.

A intel_ctrl_guest_dedicated_mask field is added to indicate the enabled counters for guest PEBS events. The IA32_PEBS_ENABLE MSR will be switched during the VMX transitions if intel_ctrl_guest_owned is set.

include/linux/perf_event.h struct perf_event { · line 754

Quoted 2 lines

include/linux/perf_event.h +2 -0

+	/* the guest specified counter index of KVM owned event, e.g PEBS */
+	int				guest_dedicated_idx;

We've always objected to guest 'owned' counters, they destroy scheduling freedom. Why are you expecting that to be any different this time?

Quoted 18 lines · Peter Zijlstra

Quoted 16 lines · Luwei Kang

From: Kan Liang <[email protected]>

The PEBS event created by host needs to be assigned specific counters requested by the guest, which means the guest and host counter indexes have to be the same or fail to create. This is needed because PEBS leaks counter indexes into the guest. Otherwise, the guest driver will be confused by the counter indexes in the status field of the PEBS record.

A guest_dedicated_idx field is added to indicate the counter index specifically requested by KVM. The dedicated event constraints would constrain the counter in the host to the same numbered counter in guest.

A intel_ctrl_guest_dedicated_mask field is added to indicate the enabled counters for guest PEBS events. The IA32_PEBS_ENABLE MSR will be switched during the VMX transitions if intel_ctrl_guest_owned is set.

Quoted 6 lines

include/linux/perf_event.h struct perf_event { · line 754

Quoted 2 lines

include/linux/perf_event.h +2 -0

+	/* the guest specified counter index of KVM owned event, e.g PEBS */
+	int				guest_dedicated_idx;

We've always objected to guest 'owned' counters, they destroy scheduling freedom. Why are you expecting that to be any different this time?

The new proposal tries to 'own' a counter by setting the event constraint. It doesn't stop other events using the counter.
If there is high priority event which requires the same counter, scheduler can still reject the request from KVM.
I don't think it destroys the scheduling freedom this time.

Quoted 28 lines · Kan

Quoted 18 lines · Peter Zijlstra

Quoted 16 lines · Luwei Kang

From: Kan Liang <[email protected]>

The PEBS event created by host needs to be assigned specific counters requested by the guest, which means the guest and host counter indexes have to be the same or fail to create. This is needed because PEBS leaks counter indexes into the guest. Otherwise, the guest driver will be confused by the counter indexes in the status field of the PEBS record.

A guest_dedicated_idx field is added to indicate the counter index specifically requested by KVM. The dedicated event constraints would constrain the counter in the host to the same numbered counter in guest.

A intel_ctrl_guest_dedicated_mask field is added to indicate the enabled counters for guest PEBS events. The IA32_PEBS_ENABLE MSR will be switched during the VMX transitions if intel_ctrl_guest_owned is set.

Quoted 6 lines

include/linux/perf_event.h struct perf_event { · line 754

Quoted 2 lines

include/linux/perf_event.h +2 -0

+	/* the guest specified counter index of KVM owned event, e.g PEBS */
+	int				guest_dedicated_idx;

We've always objected to guest 'owned' counters, they destroy scheduling freedom. Why are you expecting that to be any different this time?

Quoted 5 lines

The new proposal tries to 'own' a counter by setting the event constraint. It doesn't stop other events using the counter.
If there is high priority event which requires the same counter, scheduler can still reject the request from KVM.
I don't think it destroys the scheduling freedom this time.

Suppose your KVM thing claims counter 0/2 (ICL/SKL) for some random PEBS event, and then the host wants to use PREC_DIST.. Then one of them will be screwed for no reason what so ever.

How is that not destroying scheduling freedom? Any other situation we'd have moved the !PREC_DIST PEBS event to another counter.

Quoted 4 lines

Suppose your KVM thing claims counter 0/2 (ICL/SKL) for some random PEBS event, and then the host wants to use PREC_DIST.. Then one of them will be screwed for no reason what so ever.

The multiplexing should be triggered.

For host, if both user A and user B requires PREC_DIST, the multiplexing should be triggered for them.
Now, the user B is KVM. I don't think there is difference. The multiplexing should still be triggered. Why it is screwed?

Quoted 3 lines

How is that not destroying scheduling freedom? Any other situation we'd have moved the !PREC_DIST PEBS event to another counter.

All counters are equivalent for them. It doesn't matter if we move it to another counter. There is no impact for the user.

In the new proposal, KVM user is treated the same as other host events with event constraint. The scheduler is free to choose whether or not to assign a counter for it.

Quoted 3 lines

In the new proposal, KVM user is treated the same as other host events with event constraint. The scheduler is free to choose whether or not to assign a counter for it.

That's what it does, I understand that. I'm saying that that is creating artificial contention.

Why is this needed anyway? Can't we force the guest to flush and then move it over to a new counter?

Quoted 9 lines

Quoted 3 lines

In the new proposal, KVM user is treated the same as other host events with event constraint. The scheduler is free to choose whether or not to assign a counter for it.

That's what it does, I understand that. I'm saying that that is creating artificial contention.

Why is this needed anyway? Can't we force the guest to flush and then move it over to a new counter?

KVM only traps the MSR access. There is no MSR access during the scheduling in guest.
KVM/host only knows the request counter, when guest tries to enable the counter. It's too late for guest to start over.

Regarding to the artificial contention, as my understanding, it should rarely happen in practical.
Cloud vendors have to explicitly set pebs option in qemu to enable PEBS support for guest. They knows the environment well. They can avoid the contention. (We may implement some patches for qemu/KVM later to temporarily disable PEBS in runtime if they require.)

For now, I think we may print a warning when both host and guest require the same counter. Host can get a clue from the warning.

Quoted 3 lines · Peter Zijlstra

Quoted 3 lines

In the new proposal, KVM user is treated the same as other host events with event constraint. The scheduler is free to choose whether or not to assign a counter for it.

Quoted 6 lines

That's what it does, I understand that. I'm saying that that is creating artificial contention.

Why is this needed anyway? Can't we force the guest to flush and then move it over to a new counter?

KVM only traps the MSR access. There is no MSR access during the scheduling in guest.
KVM/host only knows the request counter, when guest tries to enable the counter. It's too late for guest to start over.

Regarding to the artificial contention, as my understanding, it should rarely happen in practical.
Cloud vendors have to explicitly set pebs option in qemu to enable PEBS support for guest. They knows the environment well. They can avoid the contention. (We may implement some patches for qemu/KVM later to temporarily disable PEBS in runtime if they require.)

For now, I think we may print a warning when both host and guest require the same counter. Host can get a clue from the warning.

Thanks,
Kan

Quoted 11 lines · Peter Zijlstra

Quoted 9 lines

Quoted 3 lines

In the new proposal, KVM user is treated the same as other host events with event constraint. The scheduler is free to choose whether or not to assign a counter for it.

That's what it does, I understand that. I'm saying that that is creating artificial contention.

Why is this needed anyway? Can't we force the guest to flush and then move it over to a new counter?

Current perf scheduling is pure software behavior. KVM only traps the MSR access. It’s impossible for KVM to impact the guest’s scheduling with current implementation.

To address the concern regarding to 'artificial contention', we have two proposals.
Could you please take a look, and share your thoughts?

Proposal 1:
Reject the guest request, if host has to use the counter which occupied by guest. At the meantime, host prints a warning.
I still think the contention should rarely happen in practical. Personally, I prefer this proposal.

Proposal 2:
Add HW advisor for the scheduler in guest.
Starts from Architectural Perfmon Version 4, IA32_PERF_GLOBAL_INUSE MSR is introduced. It provides an “InUse” bit for each programmable performance counter and fixed counter in the processor.

In perf, the scheduler will read the MSR and mask the “in used” counters. I think we can use X86_FEATURE_HYPERVISOR to limit the check in guest. For non-virtualization usage and host, nothing changed for scheduler.

But there is still a problem for this proposal. Host may request a counter later, which has been used by guest.
We can only do multiplexing or grab the counter just like proposal 1.

Quoted 11 lines

Quoted 4 lines

Suppose your KVM thing claims counter 0/2 (ICL/SKL) for some random PEBS event, and then the host wants to use PREC_DIST.. Then one of them will be screwed for no reason what so ever.

The multiplexing should be triggered.

For host, if both user A and user B requires PREC_DIST, the multiplexing should be triggered for them.
Now, the user B is KVM. I don't think there is difference. The multiplexing should still be triggered. Why it is screwed?

Becuase if KVM isn't PREC_DIST we should be able to reschedule it to a different counter.

Quoted 6 lines

Quoted 3 lines

How is that not destroying scheduling freedom? Any other situation we'd have moved the !PREC_DIST PEBS event to another counter.

All counters are equivalent for them. It doesn't matter if we move it to another counter. There is no impact for the user.

But we cannot move it to another counter, because you're pinning it.

Quoted 4 lines

In the new proposal, KVM user is treated the same as other host events with event constraint. The scheduler is free to choose whether or not to assign a counter for it.

That's what it does, I understand that. I'm saying that that is creating artificial contention.

Why is this needed anyway? Can't we force the guest to flush and then move it over to a new counter?

Quoted 15 lines

Quoted 11 lines

Quoted 4 lines

Suppose your KVM thing claims counter 0/2 (ICL/SKL) for some random PEBS event, and then the host wants to use PREC_DIST.. Then one of them will be screwed for no reason what so ever.

The multiplexing should be triggered.

For host, if both user A and user B requires PREC_DIST, the multiplexing should be triggered for them.
Now, the user B is KVM. I don't think there is difference. The multiplexing should still be triggered. Why it is screwed?

Becuase if KVM isn't PREC_DIST we should be able to reschedule it to a different counter.

Quoted 8 lines

Quoted 6 lines

Quoted 3 lines

How is that not destroying scheduling freedom? Any other situation we'd have moved the !PREC_DIST PEBS event to another counter.

All counters are equivalent for them. It doesn't matter if we move it to another counter. There is no impact for the user.

But we cannot move it to another counter, because you're pinning it.

Hi Peter,

To avoid the pinning counters, I have tried to do some evaluation about patching the PEBS record for guest in KVM. In this approach, about ~30% time increased on guest PEBS PMI handler latency ( e.g.perf record -e branch- loads:p -c 1000 ~/Tools/br_instr a).

Some implementation details as below:
1. Patching the guest PEBS records "Applicable Counters" filed when the guest required counter is not the same with the host. Because the guest PEBS driver will drop these PEBS records if the "Applicable Counters" not the same with the required counter index.
2. Traping the guest driver's behavior(VM-exit) of disabling PEBS. It happens before reading PEBS records (e.g. PEBS PMI handler, before application exit and so on)
3. To patch the Guest PEBS records in KVM, we need to get the HPA of the guest PEBS buffer.
<1> Trapping the guest write of IA32_DS_AREA register and get the GVA
of guest DS_AREA.
<2> Translate the DS AREA GVA to GPA(kvm_mmu_gva_to_gpa_read)
and get the GVA of guest PEBS buffer from DS AREA (kvm_vcpu_read_guest_atomic).
<3> Although we have got the GVA of PEBS buffer, we need to do the
address translation(GVA->GPA->HPA) for each page. Because we can't assume the GPAs of Guest PEBS buffer are always continuous.

But we met another issue about the PEBS counter reset field in DS AREA. pebs_event_reset in DS area has to be set for auto reload, which is per counter. Guest and Host may use different counters. Let's say guest wants to use counter 0, but host assign counter 1 to guest. Guest sets the reset value to pebs_event_reset[0]. However, since counter 1 is the one which is eventually scheduled, HW will use pebs_event_reset[1] as reset value.

We can't copy the value of the guest pebs_event_reset[0] to pebs_event_reset[1] directly(Patching DS AREA) because the guest driver may confused, and we can't assume the guest counter 0 and 1 are not used for this PEBS task at the same time. And what's more, KVM can't aware the guest read/write to the DS AREA because it just a general memory for guest.

What is your opinion or do you have a better proposal?

Thanks,
Luwei Kang

Quoted 11 lines

Quoted 4 lines

In the new proposal, KVM user is treated the same as other host events with event constraint. The scheduler is free to choose whether or not to assign a counter for it.

That's what it does, I understand that. I'm saying that that is creating artificial contention.

Why is this needed anyway? Can't we force the guest to flush and then move it over to a new counter?

Quoted 15 lines

Quoted 11 lines

Quoted 4 lines

Suppose your KVM thing claims counter 0/2 (ICL/SKL) for some random PEBS event, and then the host wants to use PREC_DIST.. Then one of them will be screwed for no reason what so ever.

The multiplexing should be triggered.

For host, if both user A and user B requires PREC_DIST, the multiplexing should be triggered for them.
Now, the user B is KVM. I don't think there is difference. The multiplexing should still be triggered. Why it is screwed?

Becuase if KVM isn't PREC_DIST we should be able to reschedule it to a different counter.

Quoted 8 lines

Quoted 6 lines

Quoted 3 lines

How is that not destroying scheduling freedom? Any other situation we'd have moved the !PREC_DIST PEBS event to another counter.

All counters are equivalent for them. It doesn't matter if we move it to another counter. There is no impact for the user.

But we cannot move it to another counter, because you're pinning it.

Hi Peter,

To avoid the pinning counters, I have tried to do some evaluation about patching the PEBS record for guest in KVM. In this approach, about ~30% time increased on guest PEBS PMI handler latency (
e.g.perf record -e branch-loads:p -c 1000 ~/Tools/br_instr a).

Some implementation details as below:
1. Patching the guest PEBS records "Applicable Counters" filed when the guest required counter is not the same with the host. Because the guest PEBS driver will drop these PEBS records if the "Applicable Counters" not the same with the required counter index.
2. Traping the guest driver's behavior(VM-exit) of disabling PEBS. It happens before reading PEBS records (e.g. PEBS PMI handler, before application exit and so on)
3. To patch the Guest PEBS records in KVM, we need to get the HPA of the guest PEBS buffer.
<1> Trapping the guest write of IA32_DS_AREA register and get the GVA
of guest DS_AREA.
<2> Translate the DS AREA GVA to GPA(kvm_mmu_gva_to_gpa_read)
and get the GVA of guest PEBS buffer from DS AREA (kvm_vcpu_read_guest_atomic).
<3> Although we have got the GVA of PEBS buffer, we need to do the
address translation(GVA->GPA->HPA) for each page. Because we can't assume the GPAs of Guest PEBS buffer are always continuous.

But we met another issue about the PEBS counter reset field in DS AREA. pebs_event_reset in DS area has to be set for auto reload, which is per counter. Guest and Host may use different counters. Let's say guest wants to use counter 0, but host assign counter 1 to guest. Guest sets the reset value to pebs_event_reset[0]. However, since counter 1 is the one which is eventually scheduled, HW will use pebs_event_reset[1] as reset value.

We can't copy the value of the guest pebs_event_reset[0] to pebs_event_reset[1] directly(Patching DS AREA) because the guest driver may confused, and we can't assume the guest counter 0 and 1 are not used for this PEBS task at the same time. And what's more, KVM can't aware the guest read/write to the DS AREA because it just a general memory for guest.

What is your opinion or do you have a better proposal?

Thanks,
Luwei Kang

Quoted 11 lines

Quoted 4 lines

In the new proposal, KVM user is treated the same as other host events with event constraint. The scheduler is free to choose whether or not to assign a counter for it.

That's what it does, I understand that. I'm saying that that is creating artificial contention.

Why is this needed anyway? Can't we force the guest to flush and then move it over to a new counter?

Suppose your KVM thing claims counter 0/2 (ICL/SKL) for some random PEBS event, and then the host wants to use PREC_DIST.. Then one of them will be screwed for no reason what so ever.

How is that not destroying scheduling freedom? Any other situation we'd have moved the !PREC_DIST PEBS event to another counter.

-----Original Message-----
From: kbuild test robot <[email protected]>
Sent: Saturday, March 7, 2020 12:28 AM
To: Luwei Kang <[email protected]>
Cc: kbuild-all(a)lists.01.org; x86(a)kernel.org; linux-kernel(a)vger.kernel.org; kvm(a)vger.kernel.org; peterz(a)infradead.org; mingo(a)redhat.com; acme(a)kernel.org; mark.rutland(a)arm.com; alexander.shishkin(a)linux.intel.com; jolsa(a)redhat.com; namhyung(a)kernel.org; tglx(a)linutronix.de; bp(a)alien8.de; hpa(a)zytor.com; pbonzini(a)redhat.com; sean.j.christopherson(a)intel.com; vkuznets(a)redhat.com; wanpengli(a)tencent.com; jmattson(a)google.com; joro(a)8bytes.org; pawan.kumar.gupta(a)linux.intel.com; ak(a)linux.intel.com; thomas.lendacky(a)amd.com; fenghua.yu(a)intel.com; kan.liang(a)linux.intel.com; like.xu(a)linux.intel.com
Subject: Re: [PATCH v1 05/11] KVM: x86/pmu: Add support to reprogram PEBS event for guest counters

Hi Luwei,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on kvm/linux-next] [also build test ERROR on tip/perf/core tip/auto-latest v5.6-rc4 next-20200306] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:
https://github.com/0day-ci/linux/commits/Luwei-Kang/PEBS-virtualization-ena
bling-via-DS/20200306-013049
base: https://git.kernel.org/pub/scm/virt/kvm/kvm.git linux-next
config: x86_64-randconfig-h003-20200305 (attached as .config)
compiler: gcc-7 (Debian 7.5.0-5) 7.5.0
reproduce:

        # save the attached .config to linux build tree
        make ARCH=x86_64

If you fix the issue, kindly add following tag

Reported-by kbuild test robot <[email protected]>

All errors (new ones prefixed by >>):

ld: arch/x86/kvm/pmu.o: in function `pmc_reprogram_counter':

Quoted 1 line

Quoted 1 line

arch/x86/kvm/pmu.c:199: undefined reference to

`perf_x86_pmu_unset_auto_reload'

vim +199 arch/x86/kvm/pmu.c

   101
   102	static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type,
   103					  unsigned config, bool exclude_user,
   104					  bool exclude_kernel, bool intr,
   105					  bool in_tx, bool in_tx_cp)
   106	{
   107		struct kvm_pmu *pmu = vcpu_to_pmu(pmc->vcpu);
   108		struct perf_event *event;
   109		struct perf_event_attr attr = {
   110			.type = type,
   111			.size = sizeof(attr),
   112			.pinned = true,
   113			.exclude_idle = true,
   114			.exclude_host = 1,
   115			.exclude_user = exclude_user,
   116			.exclude_kernel = exclude_kernel,
   117			.config = config,
   118			.disabled = 1,
   119		};
   120		bool pebs = test_bit(pmc->idx, (unsigned long
*)&pmu->pebs_enable);
   121
   122		attr.sample_period = (-pmc->counter) & pmc_bitmask(pmc);
   123
   124		if (in_tx)
   125			attr.config |= HSW_IN_TX;
   126		if (in_tx_cp) {
   127			/*
   128			 * HSW_IN_TX_CHECKPOINTED is not supported with

nonzero
129 * period. Just clear the sample period so at least
130 * allocating the counter doesn't fail.
131 */

   132			attr.sample_period = 0;
   133			attr.config |= HSW_IN_TX_CHECKPOINTED;
   134		}
   135
   136		if (pebs) {
   137			/*
   138			 * Host never knows the precision level set by guest.
   139			 * Force Host's PEBS event to precision level 1, which will
   140			 * not impact the accuracy of the results for guest PEBS

events.
141 * Because,
142 * - For most cases, there is no difference among precision
143 * level 1 to 3 for PEBS events.
144 * - The functions as below checks the precision level in
host.
145 * But the results from these functions in host are
replaced
146 * by guest when sampling the guest.
147 * The accuracy for guest PEBS events will not be
impacted.
148 * -- event_constraints() impacts the index of counter.
149 * The index for host event is exactly the same as guest.
150 * It's decided by guest.
151 * -- pebs_update_adaptive_cfg() impacts the value of
152 * MSR_PEBS_DATA_CFG. When guest is switched in,
153 * the MSR value will be replaced by the value from
guest.
154 * -- setup_sample () impacts the output of a PEBS
record.
155 * Guest handles the PEBS records.
156 */
157 attr.precise_ip = 1;
158 /*
159 * When the host's PMI handler completes, it's going to
160 * enter the guest and trigger the guest's PMI handler.
161 *
162 * At this moment, this function may be called by
163 * kvm_pmu_handle_event(). However the next
sample_period
164 * hasn't been determined by guest yet and the left period,
165 * which probably be 0, is used for current sample_period.
166 *
167 * In this case, perf will mistakenly treat it as non
168 * sampling events. The PEBS event will error out.
169 *
170 * Fill it with maximum period to prevent the error out.
171 * The guest PMI handler will soon reprogram the counter.
172 */
173 if (!attr.sample_period)

   174				attr.sample_period = (-1ULL) & pmc_bitmask(pmc);
   175		}
   176
   177		event = perf_event_create_kernel_counter(&attr, -1, current,
   178							 (intr || pebs) ?
   179							 kvm_perf_overflow_intr :
   180							 kvm_perf_overflow, pmc);
   181		if (IS_ERR(event)) {
   182			pr_debug_ratelimited("kvm_pmu: event creation failed %ld
for pmc->idx = %d\n",
   183				    PTR_ERR(event), pmc->idx);
   184			return;
   185		}
   186
   187		if (pebs) {
   188			event->guest_dedicated_idx = pmc->idx;
   189			/*
   190			 * For guest PEBS events, guest takes the responsibility to
   191			 * drain PEBS buffers, and load proper values to reset

counters.
192 *
193 * Host will unconditionally set auto-reload flag for PEBS
194 * events with fixed period which is not necessary. Host
should
195 * do nothing in drain_pebs() but inject the PMI into the
guest.
196 *
197 * Unset the auto-reload flag for guest PEBS events.
198 */

Quoted 1 line

199 perf_x86_pmu_unset_auto_reload(event);

200 }

   201		pmc->perf_event = event;
   202		pmc_to_pmu(pmc)->event_count++;
   203		perf_event_enable(event);
   204		clear_bit(pmc->idx, pmc_to_pmu(pmc)->reprogram_pmi);
   205	}
   206

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

BTW:
The PEBS virtualization via Intel PT patchset V1 has been posted out and the later version will base on this patchset.
https://lkml.kernel.org/r/[email protected]/

Testing:
The guest can use PEBS feature like native. e.g.

# perf record -e instructions:ppp ./br_instr a

perf report on guest:
# Samples: 2K of event 'instructions:ppp', # Event count (approx.): 1473377250
# Overhead Command Shared Object Symbol

  57.74%  br_instr  br_instr           [.] lfsr_cond
  41.40%  br_instr  br_instr           [.] cmp_end
   0.21%  br_instr  [kernel.kallsyms]  [k] __lock_acquire

perf report on host:
# Samples: 2K of event 'instructions:ppp', # Event count (approx.): 1462721386
# Overhead Command Shared Object Symbol

  57.90%  br_instr  br_instr          [.] lfsr_cond
  41.95%  br_instr  br_instr          [.] cmp_end
   0.05%  br_instr  [kernel.vmlinux]  [k] lock_acquire

Subject: Re: [PATCH v1 00/11] PEBS virtualization enabling via DS

Quoted 2 lines

Testing:
The guest can use PEBS feature like native. e.g.

Could you please add example qemu command lines too? That will make it much easier for someone to reproduce.

-Andi

Quoted 5 lines

# perf record -e instructions:ppp ./br_instr a

perf report on guest:
# Samples: 2K of event 'instructions:ppp', # Event count (approx.):

1473377250

Quoted 7 lines

# Overhead Command Shared Object Symbol

  57.74%  br_instr  br_instr           [.] lfsr_cond
  41.40%  br_instr  br_instr           [.] cmp_end
   0.21%  br_instr  [kernel.kallsyms]  [k] __lock_acquire

perf report on host:
# Samples: 2K of event 'instructions:ppp', # Event count (approx.):

1462721386

Quoted 4 lines

# Overhead Command Shared Object Symbol

  57.90%  br_instr  br_instr          [.] lfsr_cond
  41.95%  br_instr  br_instr          [.] cmp_end
   0.05%  br_instr  [kernel.vmlinux]  [k] lock_acquire