Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3092C3A59C for ; Fri, 16 Aug 2019 13:58:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B62A7206C1 for ; Fri, 16 Aug 2019 13:58:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1565963910; bh=9V4Iv8vBB+NDb+Kk4taDjhFRZypRXqcdbkdF+Hml94c=; h=From:To:Cc:Subject:Date:List-ID:From; b=XbWofDo7NHDHki7E+VWvZwORiG4unfKEgczbhTmZTg807tqGq5dZyqUVftWSqPSTT 4yDCpWo3UAsfp5XU5LWAao8XyvSWJImvkMjOSq1k6krBCR2iH8qc4K6as9mQHSicBX fTGs4woZ8fhLvtF2wR1xXbDK/44ce5VhlS5YrwT4= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727302AbfHPN6a (ORCPT ); Fri, 16 Aug 2019 09:58:30 -0400 Received: from foss.arm.com ([217.140.110.172]:57400 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727205AbfHPN6a (ORCPT ); Fri, 16 Aug 2019 09:58:30 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AAE7B360; Fri, 16 Aug 2019 06:58:29 -0700 (PDT) Received: from fuggles.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C1F503F694; Fri, 16 Aug 2019 06:58:28 -0700 (PDT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Ard Biesheuvel , Catalin Marinas , James Morse , stable@vger.kernel.org Subject: [PATCH] arm64: ftrace: Ensure module ftrace trampoline is coherent with I-side Date: Fri, 16 Aug 2019 14:57:43 +0100 Message-Id: <20190816135743.13683-1-will@kernel.org> X-Mailer: git-send-email 2.11.0 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org The initial support for dynamic ftrace trampolines in modules made use of an indirect branch which loaded its target from the beginning of a special section (e71a4e1bebaf7 ("arm64: ftrace: add support for far branches to dynamic ftrace")). Since no instructions were being patched, no cache maintenance was needed. However, later in be0f272bfc83 ("arm64: ftrace: emit ftrace-mod.o contents through code") this code was reworked to output the trampoline instructions directly into the PLT entry but, unfortunately, the necessary cache maintenance was overlooked. Add a call to __flush_icache_range() after writing the new trampoline instructions but before patching in the branch to the trampoline. Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: James Morse Cc: Fixes: be0f272bfc83 ("arm64: ftrace: emit ftrace-mod.o contents through code") Signed-off-by: Will Deacon --- arch/arm64/kernel/ftrace.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kernel/ftrace.c b/arch/arm64/kernel/ftrace.c index 1285c7b2947f..171773257974 100644 --- a/arch/arm64/kernel/ftrace.c +++ b/arch/arm64/kernel/ftrace.c @@ -73,7 +73,7 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) if (offset < -SZ_128M || offset >= SZ_128M) { #ifdef CONFIG_ARM64_MODULE_PLTS - struct plt_entry trampoline; + struct plt_entry trampoline, *dst; struct module *mod; /* @@ -106,23 +106,27 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) * to check if the actual opcodes are in fact identical, * regardless of the offset in memory so use memcmp() instead. */ - trampoline = get_plt_entry(addr, mod->arch.ftrace_trampoline); - if (memcmp(mod->arch.ftrace_trampoline, &trampoline, - sizeof(trampoline))) { - if (plt_entry_is_initialized(mod->arch.ftrace_trampoline)) { + dst = mod->arch.ftrace_trampoline; + trampoline = get_plt_entry(addr, dst); + if (memcmp(dst, &trampoline, sizeof(trampoline))) { + if (plt_entry_is_initialized(dst)) { pr_err("ftrace: far branches to multiple entry points unsupported inside a single module\n"); return -EINVAL; } /* point the trampoline to our ftrace entry point */ module_disable_ro(mod); - *mod->arch.ftrace_trampoline = trampoline; + *dst = trampoline; module_enable_ro(mod, true); - /* update trampoline before patching in the branch */ - smp_wmb(); + /* + * Ensure updated trampoline is visible to instruction + * fetch before we patch in the branch. + */ + __flush_icache_range((unsigned long)&dst[0], + (unsigned long)&dst[1]); } - addr = (unsigned long)(void *)mod->arch.ftrace_trampoline; + addr = (unsigned long)dst; #else /* CONFIG_ARM64_MODULE_PLTS */ return -EINVAL; #endif /* CONFIG_ARM64_MODULE_PLTS */ -- 2.11.0