Skip to contentSkip to conversation

perf/x86/intel: Add Branch Monitoring support

PATCH In review Megha Dey 3 participants 15 messages Nov 3, 2017 – Nov 11, 2017 1 message not in the archive

Subsystems (4)
lore
Share & tools
lore

Megha Dey

author
#0 lore raw

cover perf/x86/intel: Add Branch Monitoring support

This patchset adds support for Intel's branch monitoring feature. This feature uses heuristics to detect the occurrence of an ROP(Return Oriented Programming) or ROP like(JOP: Jump oriented programming) attack. These heuristics are based off certain performance monitoring statistics, measured dynamically over a short configurable window period. ROP is a malware trend in which the attacker can compromise a return pointer held on the stack to redirect execution to a different desired instruction.

Currently, only the Cannonlake family of Intel processors support this feature. This feature is enabled by CONFIG_PERF_EVENTS_INTEL_BM.

Once the kernel is compiled with CONFIG_PERF_EVENTS_INTEL_BM=y on a Cannonlake system, the following perf events are added which can be viewed with perf list:

  intel_bm/branch-misp/                              [Kernel PMU event]
  intel_bm/call-ret/                                 [Kernel PMU event]
  intel_bm/far-branch/                               [Kernel PMU event]
  intel_bm/indirect-branch-misp/                     [Kernel PMU event]
  intel_bm/ret-misp/                                 [Kernel PMU event]
  intel_bm/rets/                                     [Kernel PMU event]

A perf-based kernel driver has been used to monitor the occurrence of one of the 6 branch monitoring events. There are 2 counters that each can select between one of these events for evaluation over a specified instruction window size (0 to 1023). For each counter, a threshold value (0 to 127) can be configured to set a point at which an interrupt is generated. The entire system can monitor a maximum of 2 events(either from the same or different tasks) at any given time.

Apart from the kernel driver, this patchset adds CPUID of Cannonlake processors to Intel family list and the Documentation/x86/intel_bm.txt file with some information about Intel Branch monitoring.

Megha Dey (3):
x86/cpu/intel: Add Cannonlake to Intel family perf/x86/intel/bm.c: Add Intel Branch Monitoring support x86, bm: Add documentation on Intel Branch Monitoring

 Documentation/x86/intel_bm.txt      | 216 ++++++++++++
 arch/x86/events/Kconfig             |  10 +
 arch/x86/events/intel/Makefile      |   2 +
 arch/x86/events/intel/bm.c          | 635 ++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/intel-family.h |   2 +
 arch/x86/include/asm/msr-index.h    |   5 +
 include/linux/perf_event.h          |   8 +-
 kernel/events/core.c                |   8 +
 8 files changed, 885 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/x86/intel_bm.txt
 create mode 100644 arch/x86/events/intel/bm.c
Signature · 1 line
1.9.1
author
#1 lore raw

PATCH 1/3 x86/cpu/intel: Add Cannonlake to Intel family

Add CPUID of Cannonlake (CNL) processors to Intel family list.

---
 arch/x86/include/asm/intel-family.h | 2 ++
 1 file changed, 2 insertions(+)
arch/x86/include/asm/intel-family.h +2 -0
index 35a6bc4..056bd41 100644
@@ -65,6 +65,8 @@
 #define INTEL_FAM6_ATOM_DENVERTON	0x5F /* Goldmont Microserver */
 #define INTEL_FAM6_ATOM_GEMINI_LAKE	0x7A
 
+#define INTEL_FAM6_CANNONLAKE_MOBILE	0x66
+
 /* Xeon Phi */
 
 #define INTEL_FAM6_XEON_PHI_KNL		0x57 /* Knights Landing */
Signature · 1 line
1.9.1
author
#2 lore raw

PATCH 2/3 perf/x86/intel/bm.c: Add Intel Branch Monitoring support

In reply to Megha Dey

Currently, the cannonlake family of Intel processors support the branch monitoring feature. Intel's Branch monitoring feature is trying to utilize heuristics to detect the occurrence of an ROP (Return Oriented Programming) attack.

A perf-based kernel driver has been used to monitor the occurrence of one of the 6 branch monitoring events. There are 2 counters that each can select between one of these events for evaluation over a specified instruction window size (0 to 1023). For each counter, a threshold value (0 to 127) can be configured to set a point at which ROP detection event action is taken (determined by user-space). The entire system can monitor a maximum of 2 events(either from the same or different tasks) at any given time.

Apart from window_size(global) and threshold(per-counter), various sysfs entries are provided for the user to configure: guest_disable, lbr_freeze, window_cnt_sel, cnt_and_mode (all global) and mispred_evt_cnt(per-counter).

Everytime a task is scheduled out, we save current window and count associated with the event being monitored. When the task is scheduled next, we start counting from previous count associated with this event. Thus, a full context switch in this case is not necessary.

To monitor a user space application for ROP related events, perf command line can be used as follows:

perf stat -e <name of event> <application to be monitored>

eg. For the following test program (test.c) and threshold = 100 (echo 100 > /sys/devices/intel_bm/threshold)

void func(void)

{
        return;
}

void main(void)

{
        int i;

        for (i = 0; i < 128; i++) {
                func();
        }

        return;
}

perf stat -e intel_bm/rets/ ./test

Performance counter stats for './test':

                 1      intel_bm/rets/

       0.104705937 seconds time elapsed

perf returns the number of branch monitoring interrupts occurred during the execution of the user-space application.

---
 arch/x86/events/Kconfig          |  10 +
 arch/x86/events/intel/Makefile   |   2 +
 arch/x86/events/intel/bm.c       | 635 +++++++++++++++++++++++++++++++++++++++
 arch/x86/include/asm/msr-index.h |   5 +
 include/linux/perf_event.h       |   8 +-
 kernel/events/core.c             |   8 +
 6 files changed, 667 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/events/intel/bm.c
arch/x86/events/Kconfig +10 -0
index 9a7a144..40903ca 100644
@@ -9,6 +9,16 @@ config PERF_EVENTS_INTEL_UNCORE
 	Include support for Intel uncore performance events. These are
 	available on NehalemEX and more modern processors.
 
+config PERF_EVENTS_INTEL_BM
+	bool "Intel Branch Monitoring support"
+	depends on PERF_EVENTS && CPU_SUP_INTEL && PCI
+	---help---
+	  Include support for Intel Branch monitoring. This feature utilizes
+	  heuristics for detecting ROP(Return oriented programming) like
+	  attacks. These heuristics are based off certain performance
+	  monitoring statistics, measured dynamically over a short
+	  configurable window period.
+
 config PERF_EVENTS_INTEL_RAPL
 	tristate "Intel rapl performance events"
 	depends on PERF_EVENTS && CPU_SUP_INTEL && PCI
arch/x86/events/intel/Makefile +2 -0
index 3468b0c..14235ec 100644
@@ -2,6 +2,8 @@
 obj-$(CONFIG_CPU_SUP_INTEL)		+= core.o bts.o
 obj-$(CONFIG_CPU_SUP_INTEL)		+= ds.o knc.o
 obj-$(CONFIG_CPU_SUP_INTEL)		+= lbr.o p4.o p6.o pt.o
+obj-$(CONFIG_PERF_EVENTS_INTEL_BM)	+= intel-bm-perf.o
+intel-bm-perf-objs			:= bm.o
 obj-$(CONFIG_PERF_EVENTS_INTEL_RAPL)	+= intel-rapl-perf.o
 intel-rapl-perf-objs			:= rapl.o
 obj-$(CONFIG_PERF_EVENTS_INTEL_UNCORE)	+= intel-uncore.o
arch/x86/events/intel/bm.c new file +635 -0

635 lines not on this page · raw message

arch/x86/include/asm/msr-index.h +5 -0
index ab02261..f72de49 100644
@@ -294,6 +294,11 @@
 /* Alternative perfctr range with full access. */
 #define MSR_IA32_PMC0			0x000004c1
 
+/* Intel Branch Monitoring MSRs */
+#define BR_DETECT_CONTROL_MSR		0x00000350
+#define BR_DETECT_STATUS_MSR		0x00000351
+#define BR_DETECT_COUNTER_CONFIG_BASE	0x00000354
+
 /* AMD64 MSRs. Not complete. See the architecture manual for a more
    complete list. */
 
include/linux/perf_event.h +7 -1
index 8e22f24..6d929f3 100644
@@ -168,6 +168,12 @@ struct hw_perf_event {
 	 */
 	struct task_struct		*target;
 
+	struct { /* intel_bm */
+			u64 bm_ctrl;
+			u64 bm_counter_conf;
+			atomic_t bm_poll;
+	};
+
 	/*
 	 * PMU would store hardware filter configuration
 	 * here.
@@ -191,7 +197,6 @@ struct hw_perf_event {
 	 * local64_cmpxchg() such that pmu::read() can be called nested.
 	 */
 	local64_t			prev_count;
-
 	/*
 	 * The period to start the next sample with.
 	 */
@@ -512,6 +517,7 @@ typedef void (*perf_overflow_handler_t)(struct perf_event *,
  */
 #define PERF_EV_CAP_SOFTWARE		BIT(0)
 #define PERF_EV_CAP_READ_ACTIVE_PKG	BIT(1)
+#define PERF_EV_CAP_BM			BIT(2)
 
 #define SWEVENT_HLIST_BITS		8
 #define SWEVENT_HLIST_SIZE		(1 << SWEVENT_HLIST_BITS)
kernel/events/core.c +8 -0
index 9d93db8..52b3512 100644
@@ -4596,6 +4596,11 @@ static unsigned int perf_poll(struct file *file, poll_table *wait)
 
 	poll_wait(file, &event->waitq, wait);
 
+	if (event->event_caps & PERF_EV_CAP_BM) {
+		events = atomic_xchg(&event->hw.bm_poll, 0);
+		return events;
+	}
+
 	if (is_event_hup(event))
 		return events;
 
@@ -5498,6 +5503,9 @@ void perf_event_wakeup(struct perf_event *event)
 {
 	ring_buffer_wakeup(event);
 
+	if (event->event_caps & PERF_EV_CAP_BM)
+		wake_up_all(&event->waitq);
+
 	if (event->pending_kill) {
 		kill_fasync(perf_event_fasync(event), SIGIO, event->pending_kill);
 		event->pending_kill = 0;
Signature · 1 line
1.9.1
#3 lore raw

Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

Quoted 12 lines
arch/x86/events/intel/bm.c new file +12 -0
+
+static int intel_bm_event_nmi_handler(unsigned int cmd, struct pt_regs *regs)
+{
+	struct perf_event *event;
+	union bm_detect_status stat;
+	struct perf_sample_data data;
+	int i;
+	unsigned long x;
+
+	rdmsrl(BR_DETECT_STATUS_MSR, stat.raw);
+
+	if (stat.event) {

arch/x86/events/intel/bm.c line 101

Quoted 10 lines
arch/x86/events/intel/bm.c new file +10 -0
+		wrmsrl(BR_DETECT_STATUS_MSR, 0);
+		apic_write(APIC_LVTPC, APIC_DM_NMI);
+		/*
+		 * Issue wake-up to corrresponding polling event
+		 */
+		x = stat.ctrl_hit;
+		for_each_set_bit(i, &x, bm_num_counters) {
+			event = bm_counter_owner[i];
+			perf_sample_data_init(&data, 0, event->hw.last_period);
+			perf_event_overflow(event, &data, regs);

hum, it's non sampling events only right? then you don't need any of the perf_sample_data stuff.. the perf_event_overflow call is basicaly nop

arch/x86/events/intel/bm.c line 105

Quoted 4 lines
arch/x86/events/intel/bm.c new file +4 -0
+			local64_inc(&event->count);
+			atomic_set(&event->hw.bm_poll, POLLIN);
+			event->pending_wakeup = 1;
+			irq_work_queue(&event->pending);

also this is for sampling events only

seems like you only want to increment the event->count in here

thanks,
jirka

Quoted 5 lines
arch/x86/events/intel/bm.c new file +5 -0
+		}
+		return NMI_HANDLED;
+	}
+	return NMI_DONE;
+}

SNIP

#4 lore raw

RE: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

Quoted 43 lines

-----Original Message-----
From: Jiri Olsa [mailto:[email protected]]
Sent: Saturday, November 4, 2017 6:25 AM
To: Megha Dey <[email protected]>
Cc: [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yu, Yu-cheng <[email protected]>; Brown, Len <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Andrejczuk, Grzegorz <[email protected]>; Luck, Tony <[email protected]>; [email protected]; Shankar, Ravi V <[email protected]>; Dey, Megha <[email protected]>
Subject: Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

Quoted 18 lines
arch/x86/events/intel/bm.c new file +18 -0
+
+static int intel_bm_event_nmi_handler(unsigned int cmd, struct
+pt_regs *regs) {
+	struct perf_event *event;
+	union bm_detect_status stat;
+	struct perf_sample_data data;
+	int i;
+	unsigned long x;
+
+	rdmsrl(BR_DETECT_STATUS_MSR, stat.raw);
+
+	if (stat.event) {
+		wrmsrl(BR_DETECT_STATUS_MSR, 0);
+		apic_write(APIC_LVTPC, APIC_DM_NMI);
+		/*
+		 * Issue wake-up to corrresponding polling event
+		 */
+		x = stat.ctrl_hit;
Quoted 8 lines

arch/x86/events/intel/bm.c line 101

Quoted 5 lines
arch/x86/events/intel/bm.c new file +3 -0
+		for_each_set_bit(i, &x, bm_num_counters) {
+			event = bm_counter_owner[i];
+			perf_sample_data_init(&data, 0, event-

hw.last_period);

arch/x86/events/intel/bm.c new file +1 -0
+			perf_event_overflow(event, &data, regs);

hum, it's non sampling events only right? then you don't need any of the perf_sample_data stuff.. the perf_event_overflow call is basicaly nop

Yeah you are right. We were supporting sampling initially, forgot to get rid of this code. Will change this in the next version.

Quoted 9 lines

arch/x86/events/intel/bm.c line 105

Quoted 5 lines
arch/x86/events/intel/bm.c new file +4 -0
+			local64_inc(&event->count);
+			atomic_set(&event->hw.bm_poll, POLLIN);
+			event->pending_wakeup = 1;
+			irq_work_queue(&event->pending);

also this is for sampling events only

seems like you only want to increment the event->count in here

We are currently working on a library similar to libperf which user space apps could make use of instead of perf command line monitoring. This code has been added so that the user can poll on when/if an interrupt is triggered and let the user know of its occurrence. If you think there is a better way of doing this, please let me know :)

Quoted 12 lines

thanks,
jirka

Quoted 5 lines
arch/x86/events/intel/bm.c new file +5 -0
+		}
+		return NMI_HANDLED;
+	}
+	return NMI_DONE;
+}

SNIP

#5 lore raw

Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

In reply to Megha Dey

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

arch/x86/events/intel/bm.c line 219

Quoted 10 lines
arch/x86/events/intel/bm.c new file +10 -0
+	event->event_caps |= PERF_EV_CAP_BM;
+	/*
+	 * cfg contains one of the 6 possible Branch Monitoring events
+	 */
+	cfg = event->attr.config;
+	if (cfg < 0 || cfg > (BM_MAX_EVENTS - 1))
+		return -EINVAL;
+
+	if (event->attr.sample_period) /* no sampling */
+		return -EINVAL;

you can use the 'is_sampling_event' function

jirka

#6 lore raw

RE: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

Quoted 25 lines

-----Original Message-----
From: Jiri Olsa [mailto:[email protected]]
Sent: Saturday, November 4, 2017 6:25 AM
To: Megha Dey <[email protected]>
Cc: [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yu, Yu-cheng <[email protected]>; Brown, Len <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Andrejczuk, Grzegorz <[email protected]>; Luck, Tony <[email protected]>; [email protected]; Shankar, Ravi V <[email protected]>; Dey, Megha <[email protected]>
Subject: Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

Quoted 12 lines

arch/x86/events/intel/bm.c line 219

Quoted 10 lines
arch/x86/events/intel/bm.c new file +10 -0
+	event->event_caps |= PERF_EV_CAP_BM;
+	/*
+	 * cfg contains one of the 6 possible Branch Monitoring events
+	 */
+	cfg = event->attr.config;
+	if (cfg < 0 || cfg > (BM_MAX_EVENTS - 1))
+		return -EINVAL;
+
+	if (event->attr.sample_period) /* no sampling */
+		return -EINVAL;

you can use the 'is_sampling_event' function

Will make the change.

Quoted 2 lines

jirka

#7 lore raw

Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

In reply to Megha Dey

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

Quoted 4 lines
arch/x86/events/intel/bm.c new file +4 -0
+	event->hw.bm_ctrl = (bm_window_size << BM_WINDOW_SIZE_SHIFT) |
+			    (bm_guest_disable << BM_GUEST_DISABLE_SHIFT) |
+			    (bm_lbr_freeze << BM_LBR_FREEZE_SHIFT) |
+			    (bm_window_cnt_sel << BM_WINDOW_CNT_SEL_SHIFT) |

arch/x86/events/intel/bm.c line 249

Quoted 10 lines
arch/x86/events/intel/bm.c new file +10 -0
+			    (bm_cnt_and_mode << BM_CNT_AND_MODE_SHIFT) |
+								BM_ENABLE;
+	event->hw.bm_counter_conf = (bm_threshold << BM_THRESHOLD_SHIFT) |
+			(bm_mispred_evt_cnt << BM_MISPRED_EVT_CNT_SHIFT) |
+					(cfg << BM_EVENT_TYPE_SHIFT);
+
+	wrmsrl(BR_DETECT_COUNTER_CONFIG_BASE + counter_to_use,
+						event->hw.bm_counter_conf);
+	wrmsrl(BR_DETECT_STATUS_MSR, 0);
+	event->id = counter_to_use;

I think you need to add your own id under hw_perf_event::intel_bm

we use event->id as event unique id expected also in perf stat for group reading or stat record

thanks,
jirka

#8 lore raw

Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

In reply to Megha Dey

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

Quoted 5 lines
arch/x86/events/intel/bm.c new file +5 -0
+static unsigned int bm_threshold = BM_MAX_THRESHOLD;
+static unsigned int bm_mispred_evt_cnt;
+
+/* Branch monitoring counter owners */
+static struct perf_event *bm_counter_owner[2];

SNIP

arch/x86/events/intel/bm.c line 234

Quoted 13 lines
arch/x86/events/intel/bm.c new file +13 -0
+	 * Find a hardware counter for the target task
+	 */
+	for (i = 0; i < bm_num_counters; i++) {
+		if ((bm_counter_owner[i] == NULL) ||
+			(bm_counter_owner[i]->state == PERF_EVENT_STATE_DEAD)) {
+			counter_to_use = i;
+			bm_counter_owner[i] = event;
+			break;
+		}
+	}
+
+	if (counter_to_use == -1)
+		return -EBUSY;

not sure I understand, your docs says: "There are 2 8-bit counters that each.. "

so there are 2 counters per CPU? if that's corrent, isn't this check too strict then? you could have more events configured running on other CPUs for another tasks

given that we do task only events here, should bm_counter_owner be part of task, together with the limit..? I'm probably missing something..

thanks,
jirka

#9 lore raw

RE: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

Quoted 52 lines

-----Original Message-----
From: Jiri Olsa [mailto:[email protected]]
Sent: Saturday, November 4, 2017 6:26 AM
To: Megha Dey <[email protected]>
Cc: [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yu, Yu-cheng <[email protected]>; Brown, Len <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Andrejczuk, Grzegorz <[email protected]>; Luck, Tony <[email protected]>; [email protected]; Shankar, Ravi V <[email protected]>; Dey, Megha <[email protected]>
Subject: Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

Quoted 1 line

+static unsigned int bm_threshold = BM_MAX_THRESHOLD; static

unsigned

Quoted 4 lines
(quoted diff) +4 -0
+int bm_mispred_evt_cnt;
+
+/* Branch monitoring counter owners */ static struct perf_event
+*bm_counter_owner[2];

SNIP

arch/x86/events/intel/bm.c

Quoted 5 lines
arch/x86/events/intel/bm.c new file +5 -0
+	 * Find a hardware counter for the target task
+	 */
+	for (i = 0; i < bm_num_counters; i++) {
+		if ((bm_counter_owner[i] == NULL) ||
+			(bm_counter_owner[i]->state ==

PERF_EVENT_STATE_DEAD)) {

arch/x86/events/intel/bm.c line 234

Quoted 8 lines
arch/x86/events/intel/bm.c new file +8 -0
+			counter_to_use = i;
+			bm_counter_owner[i] = event;
+			break;
+		}
+	}
+
+	if (counter_to_use == -1)
+		return -EBUSY;

not sure I understand, your docs says: "There are 2 8-bit counters that each.. "

Quoted 6 lines

so there are 2 counters per CPU? if that's corrent, isn't this check too strict then? you could have more events configured running on other CPUs for another tasks

given that we do task only events here, should bm_counter_owner be part of task, together with the limit..? I'm probably missing something..

Yes you are right. Initially, we had support for 2 events(from one or 2 tasks) to be monitored for the entire system. This indeed seems very limiting. In the next patchset, I will add support for 2 events per task (This is what the hardware can support).

Quoted 3 lines

thanks,
jirka

#10 lore raw

Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

In reply to Megha Dey

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

Quoted 15 lines
arch/x86/events/intel/bm.c new file +15 -0
+
+static void intel_bm_event_update(struct perf_event *event)
+{
+	union bm_detect_status cur_stat;
+
+	rdmsrl(BR_DETECT_STATUS_MSR, cur_stat.raw);
+	local64_set(&event->hw.prev_count, (uint64_t)cur_stat.raw);
+}
+
+static void intel_bm_event_stop(struct perf_event *event, int mode)
+{
+	wrmsrl(BR_DETECT_COUNTER_CONFIG_BASE + event->id,
+	       (event->hw.bm_counter_conf & ~1));
+
+	intel_bm_event_update(event);

arch/x86/events/intel/bm.c line 179

Quoted 10 lines
arch/x86/events/intel/bm.c new file +10 -0
+}
+
+static void intel_bm_event_del(struct perf_event *event, int flags)
+{
+	intel_bm_event_stop(event, flags);
+}
+
+static void intel_bm_event_read(struct perf_event *event)
+{
+}

should you call intel_bm_event_update in here? so the read syscall gets updated data in case the case the event is active

jirka

#11 lore raw

RE: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

Quoted 39 lines

-----Original Message-----
From: Jiri Olsa [mailto:[email protected]]
Sent: Saturday, November 4, 2017 6:26 AM
To: Megha Dey <[email protected]>
Cc: [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Yu, Yu-cheng <[email protected]>; Brown, Len <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Andrejczuk, Grzegorz <[email protected]>; Luck, Tony <[email protected]>; [email protected]; Shankar, Ravi V <[email protected]>; Dey, Megha <[email protected]>
Subject: Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

Quoted 14 lines
arch/x86/events/intel/bm.c new file +14 -0
+
+static void intel_bm_event_update(struct perf_event *event) {
+	union bm_detect_status cur_stat;
+
+	rdmsrl(BR_DETECT_STATUS_MSR, cur_stat.raw);
+	local64_set(&event->hw.prev_count, (uint64_t)cur_stat.raw); }
+
+static void intel_bm_event_stop(struct perf_event *event, int mode) {
+	wrmsrl(BR_DETECT_COUNTER_CONFIG_BASE + event->id,
+	       (event->hw.bm_counter_conf & ~1));
+
+	intel_bm_event_update(event);
+}
+
Quoted 8 lines

arch/x86/events/intel/bm.c

Quoted 5 lines
arch/x86/events/intel/bm.c new file +5 -0
+static void intel_bm_event_del(struct perf_event *event, int flags) {
+	intel_bm_event_stop(event, flags);
+}
+
+static void intel_bm_event_read(struct perf_event *event) { }

should you call intel_bm_event_update in here? so the read syscall gets updated data in case the case the event is active

We do not update the event->count in the intel_bm_event_update function. We are basically saving the status register contents when a task is being scheduled out. So it has nothing to do with the read syscall. Having said that, I will probably put what stop() does in del() and get rid of the stop() function.

Quoted 2 lines

jirka

#12 lore raw

Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

In reply to Megha Dey

On Fri, Nov 03, 2017 at 11:00:05AM -0700, Megha Dey wrote:

SNIP

Quoted 6 lines
arch/x86/events/intel/bm.c new file +6 -0
+
+static int intel_bm_event_nmi_handler(unsigned int cmd, struct pt_regs *regs)
+{
+	struct perf_event *event;
+	union bm_detect_status stat;
+	struct perf_sample_data data;

arch/x86/events/intel/bm.c line 95

Quoted 10 lines
arch/x86/events/intel/bm.c new file +10 -0
+	int i;
+	unsigned long x;
+
+	rdmsrl(BR_DETECT_STATUS_MSR, stat.raw);
+
+	if (stat.event) {
+		wrmsrl(BR_DETECT_STATUS_MSR, 0);
+		apic_write(APIC_LVTPC, APIC_DM_NMI);
+		/*
+		 * Issue wake-up to corrresponding polling event

s/rrr/rr/ ;-)

jirka

A message that is not in the archive (sent off-list or not yet received): [email protected]

#14 lore raw

Re: [PATCH V0 2/3] perf/x86/intel/bm.c: Add Intel Branch Monitoring support

Quoted 6 lines · Alexander Shishkin
Quoted 2 lines · Megha Dey
arch/x86/events/intel/bm.c new file +2 -0
+static int intel_bm_event_init(struct perf_event *event)
+{

...

Quoted 13 lines

arch/x86/events/intel/bm.c line 229

Quoted 9 lines
arch/x86/events/intel/bm.c new file +9 -0
+	/*
+	 * Find a hardware counter for the target task
+	 */
+	for (i = 0; i < bm_num_counters; i++) {
+		if ((bm_counter_owner[i] == NULL) ||
+			(bm_counter_owner[i]->state == PERF_EVENT_STATE_DEAD)) {
+			counter_to_use = i;
+			bm_counter_owner[i] = event;
+			break;

How are two concurrent perf_event_open()s not going to race here? Also, I'm not sure what's the value of looking at the ->state here. Shouldn't the ->destroy() method clear the corresponding array slot?

Yes you are right. I will add a locking mechanism here to prevent racing and remove the ->state in the next version.

Quoted 6 lines
Quoted 3 lines
(quoted diff) +2 -0
+		}
+	}

...

Quoted 7 lines

arch/x86/events/intel/bm.c line 248

Quoted 3 lines
arch/x86/events/intel/bm.c new file +3 -0
+	wrmsrl(BR_DETECT_COUNTER_CONFIG_BASE + counter_to_use,
+						event->hw.bm_counter_conf);
+	wrmsrl(BR_DETECT_STATUS_MSR, 0);

These wrmsrs will happen on whatever CPU perf_event_open() is called on, as opposed to the CPU where the event will be scheduled. You probably want to keep the MSR accesses in the start()/stop() callbacks.

Agreed, don't think we need this code here. We are writing to the MSRs in start() anyways.

Quoted 5 lines

Regards,
--
Alex

author
#15 lore raw

PATCH 3/3 x86, bm: Add documentation on Intel Branch Monitoring

In reply to Megha Dey

This patch adds the Documentation/x86/intel_bm.txt file with some information about Intel Branch monitoring.

---
 Documentation/x86/intel_bm.txt | 216 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 216 insertions(+)
 create mode 100644 Documentation/x86/intel_bm.txt
Documentation/x86/intel_bm.txt new file +216 -0
new file mode 100644
index 0000000..25b7177
@@ -0,0 +1,216 @@
+Intel(R) Branch Monitoring
+
+Copyright (C) 2017 Intel Corporation
+
+Megha Dey <[email protected]>
+Yu-Cheng Yu <[email protected]>
+
+I. Overview
+===========
+
+The Cannonlake family of Intel processors support the branch monitoring
+feature. This feature uses heuristics to detect the occurrence of an ROP
+(Return Oriented Programming) or ROP like(JOP:Jump oriented programming)
+attack. These heuristics are based off certain performance monitoring
+statistics, measured dynamically over a short configurable window period.
+ROP is a malware trend in which the attacker can compromise a return
+pointer held on the stack to redirect execution to a different desired
+instruction.
+
+Support for branch monitoring has been added via Linux kernel perf event
+infrastructure. This feature is enabled by CONFIG_PERF_EVENTS_INTEL_BM.
+
+Once the kernel is compiled with CONFIG_PERF_EVENTS_INTEL_BM=y on a
+Cannonlake system, the following perf events are added which can be viewed
+with perf list:
+  intel_bm/branch-misp/                              [Kernel PMU event]
+  intel_bm/call-ret/                                 [Kernel PMU event]
+  intel_bm/far-branch/                               [Kernel PMU event]
+  intel_bm/indirect-branch-misp/                     [Kernel PMU event]
+  intel_bm/ret-misp/                                 [Kernel PMU event]
+  intel_bm/rets/                                     [Kernel PMU event]
+
+II. Hardware details
+====================
+
+The MSRs associated with branch monitoring are as follows:
+
+1. BR_DETECT_CTRL : Branch Monitoring Global control
+   Used for enabling and configuring global capability
+
+2. BR_DETECT_STATUS : Branch Monitoring Global Status
+   Used by SW handler for determining detect status
+
+3. BR_DETECT_COUNTER_CONFIG_i : Branch Monitoring Counter Configuration
+   Per-cpu branch monitoring counter Configuration
+
+There are 2 8-bit counters that each can select between one of the
+following 6 events:
+
+1. RET instructions: Counts the number of near return instructions retired
+
+2. CALL-RET instructions: Counts the difference between the number of near
+   return and call instructions retired
+
+3. RET mispredicts: Mispredicted return instructions retired
+
+4. Branch (all) mispredicts: Counts the number of mispredicted branches
+
+5. Indirect branch mispredicts: Counts the number of mispredicted indirect
+   near branch instructions. Includes indirect near jump/call instructions
+
+6. Far branch instructions: Counts the number of far branches retired
+
+Branch Monitoring hardware utilizes various existing performance related
+counter events. Of the 6 events above, only call-ret is newly implemented.
+
+The events are evaluated over a specified 10-bit instruction window size
+(0 to 1023). For each counter, a threshold value (0 to 127) can be
+configured to set a point at which an interrupt is generated and a
+detection event action is taken (determined by user-space). This can take
+the form of signaling an interrupt and/or freezing the state of the last
+branch record information.
+
+The event counters are reset after every 'window size' instructions by the
+hardware.
+
+The feature is for user mode (privilege level > 0) operation only, which is
+the known malware security threat target environment. While in supervisor
+mode, this heuristic detection counter activity is suspended. This behavior
+(user mode) is independent of root vs. non-root with respect to
+virtualization technology execution.
+
+III. Software Implementation
+============================
+
+A perf-based kernel driver has been used to monitor the occurrence of
+one of the 6 branch monitoring events.
+
+If an branch monitoring interrupt is generated, the interrupt bit is set
+which is cleared by interrupt handler and the event counters are reset.
+
+The entire system can monitor a maximum of 2 events at any given time.
+These events can belong to the same or different tasks.
+
+Everytime a task is scheduled out, we save current window and count
+associated with the event being monitored. When the task is scheduled next,
+we start counting from previous count associated with this event. Thus, a
+full context switch in this case is not necessary.
+
+The Branch Monitoring exception can be configured as a regular interrupt or
+an NMI. We chain an NMI handler after PMU, because
+1. It will not interfere with PMU events
+2. We only monitor for user-mode events, and this will not delay branch
+   monitoring events for user-mode
+
+We monitor only per-task events. It does not make sense to monitor all tasks
+for an attack. This could generate a lot of false positives.
+
+IV. User-configurable inputs
+============================
+
+Several sysfs entries are provided in /sys/devices/intel_bm/ to configure
+controls for the supported hardware heuristics.
+
+1. LBR freeze: /sys/devices/intel-bm/lbr_freeze
+   possible values are 0 or 1. By default this is disabled(0). When enabled,
+   an LBR freeze is observed on threshold trip
+
+2. Guest Disable: /sys/devices/intel-bm/guest_disable
+   Possible values are 0 or 1. By default it is 0. When set to ‘1’, branch
+   monitoring feature is disabled when operating at VMX non-root operation.
+
+3. Window size: /sys/devices/intel-bm/window_size
+   By default, window size is 1023. It can take values from 0 to 1023. This
+   represents the number of instructions to be executed before the event
+   counters are reset.
+
+4. Window count select: /sys/devices/intel-bm/window_cnt_sel
+   Possible values are:
+   ‘00 = instructions retired
+   ‘01 = branches retired
+   ‘10 = returned instructions retired
+   ‘11 = indirect branch instructions retired
+   By default, it has a value of 0.
+
+5. Count and mode: /sys/devices/intel-bm/cnt_and_mode
+   Possible values are 0 or 1. By default it is 0. When set to ‘1’, the
+   overall event triggering condition is true only if both enabled
+   counter’s threshold conditions are true. When ‘0’, the threshold
+   tripping condition is true if either enabled counter’s threshold is
+   true. If a counter is not enabled, then it does not factor into the
+   AND’ing logic
+
+6. Threshold: /sys/devices/intel-bm/threshold
+   An unsigned value of 0 to 127 is supported. The value 0 of counter
+   threshold will result in branch monitoring event signaled after every
+   instruction. By default, it has a value of 127.
+
+7. Mispredict counting behaviour: /sys/devices/intel-bm/mispred_evt_cnt
+   Possible values are:
+   0 = mispredict events are counted in a window
+   1 = mispredict events are counted based on a consecutive occurrence.
+   By default, it has a value of 0.
+
+Threshold and Mispredict events counting behaviour are per-counter
+configurations whereas the rest are global.
+
+V. Example usage
+================
+
+1. To monitor a user space application for branch monitoring events, perf
+command line can be used as follows:
+
+perf stat -e intel_bm/rets/ ./test
+
+ Performance counter stats for './test':
+
+                 1      intel_bm/rets/
+
+       0.104705937 seconds time elapsed
+
+where test.c is:
+
+void func(void)
+{
+        return;
+}
+
+void main(void)
+{
+        int i;
+
+        for (i = 0; i < 128; i++) {
+                func();
+        }
+
+        return;
+}
+
+and threshold = 100 (echo 100 > /sys/devices/intel_bm/threshold)
+
+perf returns the number of branch monitoring interrupts occurred when the
+user-space application was running.
+
+2. To monitor 2 events for a task,
+
+perf stat -e intel_bm/far-branch/,intel_bm/rets/ ./rets-128.bin
+
+ Performance counter stats for './rets-128.bin':
+
+                 0      intel_bm/far-branch/
+                 1      intel_bm/rets/
+
+       0.104057608 seconds time elapsed
+
+For the above example, the threshold and window size are shared.
+
+3. To monitor 2 events with different thresholds(same or different task)
+
+On terminal 1:
+echo <threshold1> > /sys/devices/intel_bm/threshold
+perf stat -e intel_bm/rets/ ./test.bin
+
+On terminal 2:
+echo <threshold2> > /sys/devices/intel_bm/threshold
+perf stat -e intel_bm/call-ret/ ./test.bin
Signature · 1 line
1.9.1