Skip to contentSkip to conversation

unrecognizable insn generated in plugin?

PATCH Tycho Andersen 3 participants 5 messages May 30, 2019 – May 31, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi all,

I've been trying to implement an idea Andy suggested recently for preventing some kinds of ROP attacks. The discussion of the idea is here:
https://lore.kernel.org/linux-mm/[email protected]/

Right now I'm struggling to get my plugin to compile without crashing. The basic idea is to insert some code before every "pop rbp" and "pop rsp"; I've figured out how to find these instructions, and I'm inserting code using:

emit_insn(gen_rtx_XOR(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),

                      gen_rtx_MEM(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM))));

The plugin completes successfully, but GCC complains later,

kernel/seccomp.c: In function ‘seccomp_check_filter’:
kernel/seccomp.c:242:1: error: unrecognizable insn:
}
^
(insn 698 645 699 17 (xor:DI (reg:DI 6 bp)

        (mem:DI (reg:DI 6 bp) [0  S8 A8])) "kernel/seccomp.c":242 -1
     (nil))

during RTL pass: shorten
kernel/seccomp.c:242:1: internal compiler error: in insn_min_length, at config/i386/i386.md:14714

I assume this is because some internal metadata is screwed up, but I have no clue as to what that is or how to fix it. My gcc version is 8.3.0, and config/i386/i386.md:14714 of that tag looks mostly unrelated.

I had problems earlier because I was trying to run it after *clean_state which is the thing that does init_insn_lengths(), but now I'm running it after *stack_regs, so I thought it should be ok...

Anyway, the full plugin draft is below. You can run it by adding CONFIG_GCC_PLUGIN_HEAPLEAP=y to your kernel config.

Thanks!

Tycho

Quoted 1 line

From 83b0631f14784ce11362ebd64e40c8d25c0decee Mon Sep 17 00:00:00 2001

From: Tycho Andersen <[email protected]>
Date: Fri, 19 Apr 2019 19:24:58 -0600
Subject: [PATCH] heapleap

---
 scripts/Makefile.gcc-plugins          |   8 ++
 scripts/gcc-plugins/Kconfig           |   4 +
 scripts/gcc-plugins/heapleap_plugin.c | 189 ++++++++++++++++++++++++++
 3 files changed, 201 insertions(+)
scripts/Makefile.gcc-plugins +8 -0
index 5f7df50cfe7a..283b81dc5742 100644
@@ -44,6 +44,14 @@ ifdef CONFIG_GCC_PLUGIN_ARM_SSP_PER_TASK
 endif
 export DISABLE_ARM_SSP_PER_TASK_PLUGIN
 
+gcc-plugin-$(CONFIG_GCC_PLUGIN_HEAPLEAP)	+= heapleap_plugin.so
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_HEAPLEAP)		\
+			+= -DHEAPLEAP_PLUGIN
+ifdef CONFIG_GCC_PLUGIN_HEAPLEAP
+    DISABLE_HEAPLEAP_PLUGIN += -fplugin-arg-heapleap_plugin-disable
+endif
+export DISABLE_HEAPLEAP_PLUGIN
+
 # All the plugin CFLAGS are collected here in case a build target needs to
 # filter them out of the KBUILD_CFLAGS.
 GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y))
scripts/gcc-plugins/Kconfig +4 -0
index 74271dba4f94..491b9cd5df1a 100644
@@ -226,4 +226,8 @@ config GCC_PLUGIN_ARM_SSP_PER_TASK
 	bool
 	depends on GCC_PLUGINS && ARM
 
+config GCC_PLUGIN_HEAPLEAP
+	bool "Prevent 'pop esp' type instructions from loading an address in the heap"
+	depends on GCC_PLUGINS
+
 endif
scripts/gcc-plugins/heapleap_plugin.c new file +189 -0

189 lines not on this page · raw message

Signature · 1 line
2.20.1
#1 lore raw
Quoted 8 lines · Tycho Andersen

Hi all,

I've been trying to implement an idea Andy suggested recently for preventing some kinds of ROP attacks. The discussion of the idea is here:
https://lore.kernel.org/linux-mm/[email protected]/

Quoted 6 lines

Right now I'm struggling to get my plugin to compile without crashing. The basic idea is to insert some code before every "pop rbp" and "pop rsp"; I've figured out how to find these instructions, and I'm inserting code using:

emit_insn(gen_rtx_XOR(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),

                      gen_rtx_MEM(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM))));

Simplely this xor does not set anything.
I think you want something like:
emit_insn(gen_rtx_SET(gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),
gen_rtx_XOR(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),

      gen_rtx_MEM(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM)))));

But that might not work either, you might need some thing more.

Thanks,
Andrew Pinski

Quoted 271 lines

The plugin completes successfully, but GCC complains later,

kernel/seccomp.c: In function ‘seccomp_check_filter’:
kernel/seccomp.c:242:1: error: unrecognizable insn:
}
^
(insn 698 645 699 17 (xor:DI (reg:DI 6 bp)

        (mem:DI (reg:DI 6 bp) [0  S8 A8])) "kernel/seccomp.c":242 -1
     (nil))

during RTL pass: shorten
kernel/seccomp.c:242:1: internal compiler error: in insn_min_length, at config/i386/i386.md:14714

I assume this is because some internal metadata is screwed up, but I have no clue as to what that is or how to fix it. My gcc version is 8.3.0, and config/i386/i386.md:14714 of that tag looks mostly unrelated.

I had problems earlier because I was trying to run it after *clean_state which is the thing that does init_insn_lengths(), but now I'm running it after *stack_regs, so I thought it should be ok...

Anyway, the full plugin draft is below. You can run it by adding CONFIG_GCC_PLUGIN_HEAPLEAP=y to your kernel config.

Thanks!

Tycho

From 83b0631f14784ce11362ebd64e40c8d25c0decee Mon Sep 17 00:00:00 2001
From: Tycho Andersen <[email protected]>
Date: Fri, 19 Apr 2019 19:24:58 -0600
Subject: [PATCH] heapleap

---
 scripts/Makefile.gcc-plugins          |   8 ++
 scripts/gcc-plugins/Kconfig           |   4 +
 scripts/gcc-plugins/heapleap_plugin.c | 189 ++++++++++++++++++++++++++
 3 files changed, 201 insertions(+)
scripts/Makefile.gcc-plugins +8 -0
index 5f7df50cfe7a..283b81dc5742 100644
@@ -44,6 +44,14 @@ ifdef CONFIG_GCC_PLUGIN_ARM_SSP_PER_TASK
 endif
 export DISABLE_ARM_SSP_PER_TASK_PLUGIN
 
+gcc-plugin-$(CONFIG_GCC_PLUGIN_HEAPLEAP)       += heapleap_plugin.so
+gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_HEAPLEAP)                \
+                       += -DHEAPLEAP_PLUGIN
+ifdef CONFIG_GCC_PLUGIN_HEAPLEAP
+    DISABLE_HEAPLEAP_PLUGIN += -fplugin-arg-heapleap_plugin-disable
+endif
+export DISABLE_HEAPLEAP_PLUGIN
+
 # All the plugin CFLAGS are collected here in case a build target needs to
 # filter them out of the KBUILD_CFLAGS.
 GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y))
scripts/gcc-plugins/Kconfig +4 -0
index 74271dba4f94..491b9cd5df1a 100644
@@ -226,4 +226,8 @@ config GCC_PLUGIN_ARM_SSP_PER_TASK
        bool
        depends on GCC_PLUGINS && ARM
 
+config GCC_PLUGIN_HEAPLEAP
+       bool "Prevent 'pop esp' type instructions from loading an address in the heap"
+       depends on GCC_PLUGINS
+
 endif
scripts/gcc-plugins/heapleap_plugin.c new file +189 -1
new file mode 100644
index 000000000000..5051b96d79f4
@@ -0,0 +1,189 @@
+/*
+ * This is based on an idea from Andy Lutomirski described here:
+ * https://lore.kernel.org/linux-mm/[email protected]/
+ *
+ * unsigned long offset = *rsp - rsp;
+ * offset >>= THREAD_SHIFT;
+ * if (unlikely(offset))
+ *     BUG();
+ * POP RSP;
+ */
+
+#include "gcc-common.h"
+
+__visible int plugin_is_GPL_compatible;
+static bool disable = false;
+
+static struct plugin_info heapleap_plugin_info = {
+       .version = "1",
+       .help = "disable\t\tdo not activate the plugin\n"
+};
+
+static bool heapleap_gate(void)
+{
+       tree section;
+
+       /*
+        * Similar to stackleak, we only do this for user code for now.
+        */
+       section = lookup_attribute("section",
+                                  DECL_ATTRIBUTES(current_function_decl));
+       if (section && TREE_VALUE(section)) {
+               section = TREE_VALUE(TREE_VALUE(section));
+
+               if (!strncmp(TREE_STRING_POINTER(section), ".init.text", 10))
+                       return false;
+               if (!strncmp(TREE_STRING_POINTER(section), ".devinit.text", 13))
+                       return false;
+               if (!strncmp(TREE_STRING_POINTER(section), ".cpuinit.text", 13))
+                       return false;
+               if (!strncmp(TREE_STRING_POINTER(section), ".meminit.text", 13))
+                       return false;
+       }
+
+       return !disable;
+}
+
+/*
+ * Check that:
+ *
+ * unsigned long offset = *rbp - rbp;
+ * offset >>= THREAD_SHIFT;
+ * if (unlikely(offset))
+ *     BUG();
+ * pop rbp;
+ *
+ * (we should probably do the same for rsp?)
+ */
+static void heapleap_add_check(rtx_insn *insn)
+{
+       rtx_insn *seq_head;
+
+       fprintf(stderr, "adding heapleap check\n");
+       print_rtl_single(stderr, insn);
+
+       start_sequence();
+
+       /* xor ebp [ebp] */
+       emit_insn(gen_rtx_XOR(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),
+                             gen_rtx_MEM(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM))));
+
+       /* ebp >> THREAD_SHIFT */
+       /*
+        * TODO: THREAD_SHIFT isn't defined for every arch, including x86.
+        * THREAD_SIZE for x86_64 is 4096 * 2, so THREAD_SHIFT would be 13
+        * there. We should at least compute this from THREAD_SIZE though.
+        */
+       emit_insn(gen_rtx_LSHIFTRT(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),
+                                  GEN_INT(13)));
+
+       /*
+        * We're inserting right before the final pass, and we're adding some
+        * kind of jump, thus splitting the basic block that is the epilogue.
+        * That probably causes problems, and currently gcc crashes when doing
+        * the final pass after we emit this, so we probably need to do better.
+        */
+       emit_insn(gen_rtx_IF_THEN_ELSE(DImode,
+                       gen_rtx_NE(DImode,
+                               gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),
+                               GEN_INT(0)),
+                       /*
+                        * we're really not supposed to BUG() for this stuff;
+                        * maybe we should figure out how to call WARN()? might
+                        * be painful.
+                        */
+                       gen_ud2(),
+                       /* poor man's no-op, i.e. how do i do this better? */
+                       gen_rtx_SET(gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),
+                                   gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM))));
+       seq_head = get_insns();
+       end_sequence();
+
+       emit_insn_before(seq_head, insn);
+}
+
+static unsigned int heapleap_execute(void)
+{
+       rtx_insn *insn, *next;
+
+       if (strcmp(IDENTIFIER_POINTER(DECL_NAME(cfun->decl)), "seccomp_check_filter"))
+               return 0;
+
+       for (insn = get_insns(); insn; insn = next) {
+               rtx body, set, lhs, rhs;
+               int i;
+
+               next = NEXT_INSN(insn);
+               if (!next)
+                       continue;
+
+               if (!RTX_FRAME_RELATED_P(next) || !NONJUMP_INSN_P(next))
+                       continue;
+
+               /*
+                * I don't understand why we need this; but PATTERN(insn) is a
+                * CODE_LABEL, so...
+                */
+               body = XEXP(insn, 1);
+               set = PATTERN(body);
+               if (GET_CODE(set) != SET)
+                       continue;
+
+               /* TODO: use SET_DEST() here instead? */
+               lhs = XEXP(set, 0);
+               /* TODO: ebp vs esp? esp only occurs twice in my linked kernel */
+               if (GET_CODE(lhs) != REG || REGNO(lhs) != HARD_FRAME_POINTER_REGNUM)
+                       continue;
+
+               /* TODO: use SET_SRC() here instead? */
+               rhs = XEXP(set, 1);
+               if (GET_CODE(rhs) != MEM)
+                       continue;
+
+               heapleap_add_check(next);
+       }
+
+       return 0;
+}
+
+#define PASS_NAME heapleap
+#include "gcc-generate-rtl-pass.h"
+
+__visible int plugin_init(struct plugin_name_args *plugin_info,
+                         struct plugin_gcc_version *version)
+{
+       const char * const plugin_name = plugin_info->base_name;
+       const int argc = plugin_info->argc;
+       const struct plugin_argument * const argv = plugin_info->argv;
+       int i;
+
+       /*
+        * *clean_state is the pass that does init_insn_lengths(), so we can't
+        * do anything after this, because gcc fails there's not a length for
+        * every instruction in the final pass
+        */
+       PASS_INFO(heapleap, "*stack_regs", 1, PASS_POS_INSERT_AFTER);
+
+       if (!plugin_default_version_check(version, &gcc_version)) {
+               error(G_("incompatible gcc/plugin versions"));
+               return 1;
+       }
+
+       for (i = 0; i < argc; i++) {
+               if (!strcmp(argv[i].key, "disable")) {
+                       disable = true;
+                       return 0;
+               } else {
+                       error(G_("unknown option '-fplugin-arg-%s-%s'"),
+                                       plugin_name, argv[i].key);
+                       return 1;
+               }
+       }
+
+       register_callback(plugin_name, PLUGIN_INFO, NULL,
+                                               &heapleap_plugin_info);
+
+       register_callback(plugin_name, PLUGIN_PASS_MANAGER_SETUP, NULL,
+                                       &heapleap_pass_info);
+       return 0;
+}
--

2.20.1

#2 lore raw

Hi Andrew,

Quoted 16 lines · Andrew Pinski
Quoted 8 lines · Tycho Andersen

Hi all,

I've been trying to implement an idea Andy suggested recently for preventing some kinds of ROP attacks. The discussion of the idea is here:
https://lore.kernel.org/linux-mm/[email protected]/

Quoted 6 lines

Right now I'm struggling to get my plugin to compile without crashing. The basic idea is to insert some code before every "pop rbp" and "pop rsp"; I've figured out how to find these instructions, and I'm inserting code using:

emit_insn(gen_rtx_XOR(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),

                      gen_rtx_MEM(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM))));
Quoted 7 lines

Simplely this xor does not set anything.
I think you want something like:
emit_insn(gen_rtx_SET(gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),
gen_rtx_XOR(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM),

      gen_rtx_MEM(DImode, gen_rtx_REG(DImode, HARD_FRAME_POINTER_REGNUM)))));

But that might not work either, you might need some thing more.

Yes, thanks. You're also right that I still see the same problem:

kernel/seccomp.c: In function ‘seccomp_check_filter’:
kernel/seccomp.c:242:1: error: unrecognizable insn:
}
^
(insn 698 645 699 17 (set (reg:DI 6 bp)

        (xor:DI (reg:DI 6 bp)
            (mem:DI (reg:DI 6 bp) [0  S8 A8]))) "kernel/seccomp.c":242 -1
     (nil))

during RTL pass: shorten
kernel/seccomp.c:242:1: internal compiler error: in insn_min_length, at config/i386/i386.md:14714

Thanks,

Tycho

#3 lore raw
Quoted 13 lines · Tycho Andersen

Hi Andrew,

Quoted 9 lines · Andrew Pinski
Quoted 8 lines · Tycho Andersen

Hi all,

I've been trying to implement an idea Andy suggested recently for preventing some kinds of ROP attacks. The discussion of the idea is here:
https://lore.kernel.org/linux-mm/[email protected]/

Hi Tycho,

I realise this is maybe not relevant to the topic of fixing the plugin; but I'm struggling to understand what this is intending to protect against.

The idea seems to be to make sure that restored rbp, rsp values are "close" to the current rbp, rsp values? The only scenario I can see this providing any benefit is if an attacker can only corrupt a saved stack/frame pointer, which seems like such an unlikely situation that it's not really worth adding any complexity to defend against.

An attacker who has control of rip can surely get a controlled value into rsp in various ways; a quick scan of the current Ubuntu 18.04 kernel image offers the following sequence (which shows up everywhere):

lea rsp, qword ptr [r10 - 8]
ret

I'd assume that it's not tremendously difficult for an attacker to chain to this without needing to previously pivot out the stack pointer, assuming that at the point at which they gain control of rip they have control over some state somewhere. If you could explain the exact attack scenario that you have in mind then perhaps I could provide a better explanation of how one might bypass it.

Regards,
Mark

#4 lore raw
Quoted 22 lines · Mark Brand
Quoted 13 lines · Tycho Andersen

Hi Andrew,

Quoted 9 lines · Andrew Pinski
Quoted 8 lines · Tycho Andersen

Hi all,

I've been trying to implement an idea Andy suggested recently for preventing some kinds of ROP attacks. The discussion of the idea is here:
https://lore.kernel.org/linux-mm/[email protected]/

Hi Tycho,

I realise this is maybe not relevant to the topic of fixing the plugin; but I'm struggling to understand what this is intending to protect against.

Quoted 8 lines

The idea seems to be to make sure that restored rbp, rsp values are "close" to the current rbp, rsp values? The only scenario I can see this providing any benefit is if an attacker can only corrupt a saved stack/frame pointer, which seems like such an unlikely situation that it's not really worth adding any complexity to defend against.

An attacker who has control of rip can surely get a controlled value into rsp in various ways;

Yes, if you already have control of rip this doesn't help you.

Quoted 7 lines

a quick scan of the current Ubuntu 18.04
kernel image offers the following sequence (which shows up everywhere):

lea rsp, qword ptr [r10 - 8]
ret

Quoted 6 lines

I'd assume that it's not tremendously difficult for an attacker to chain to this without needing to previously pivot out the stack pointer, assuming that at the point at which they gain control of rip they have control over some state somewhere. If you could explain the exact attack scenario that you have in mind then perhaps I could provide a better explanation of how one might bypass it.

The core bit that's important here is the writes to rsp/rbp, not the fact that they're pop instructions. The insight is that we know how the thread's stack should be aligned, and so any value that's written to these registers outside of that alignment (during "normal" execution) is a bug.

The idea is that a ROP attack requires a payload to be injected somewhere so that it can return to this payload and execute this attack. This is most probably done via some allocation elsewhere (add_key() or unreceived data or whatever) since as you note, the stack should be mostly well protected.

So then, if we don't allow anyone to write anything that's not on the stack to rsp/rbp, in principle we should be safe from ROP attacks where the payload is elsewhere. As you note, preventing bad "pop rpb" instructions is not enough, nor is preventing bad "pop rsp", as Andy initially proposed. We need to prevent all bad writes to these registers, including the sequence you mentioned, and presumably others. So there would need to be a lot more matching and checks inserted, and maybe it would ultimately be slow. Right now I just wanted to play with it for a bit to see if I can get it to work at all even in one case :)

Am I thinking about this wrong? Any discussion is welcome, thanks!

Tycho