Date:   Fri, 5 Aug 2022 16:58:08 -0400
From:   Joe Lawrence <joe.lawrence@redhat.com>
To:     Song Liu <song@kernel.org>
Cc:     live-patching@vger.kernel.org, jpoimboe@kernel.org,
        jikos@kernel.org, mbenes@suse.cz, pmladek@suse.com,
        kernel-team@fb.com, Josh Poimboeuf <jpoimboe@redhat.com>
Subject: Re: [PATCH v4] livepatch: Clear relocation targets on a module
 removal
Message-ID: <Yu2EYG0YjPLjiPk0@redhat.com>
References: <20220801212129.2008177-1-song@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20220801212129.2008177-1-song@kernel.org>
Precedence: bulk

On Mon, Aug 01, 2022 at 02:21:29PM -0700, Song Liu wrote:
> From: Miroslav Benes <mbenes@suse.cz>
> 
> Josh reported a bug:
> 
>   When the object to be patched is a module, and that module is
>   rmmod'ed and reloaded, it fails to load with:
> 
>   module: x86/modules: Skipping invalid relocation target, existing value is nonzero for type 2, loc 00000000ba0302e9, val ffffffffa03e293c
>   livepatch: failed to initialize patch 'livepatch_nfsd' for module 'nfsd' (-8)
>   livepatch: patch 'livepatch_nfsd' failed for module 'nfsd', refusing to load module 'nfsd'
> 
>   The livepatch module has a relocation which references a symbol
>   in the _previous_ loading of nfsd. When apply_relocate_add()
>   tries to replace the old relocation with a new one, it sees that
>   the previous one is nonzero and it errors out.
> 
>   On ppc64le, we have a similar issue:
> 
>   module_64: livepatch_nfsd: Expected nop after call, got e8410018 at e_show+0x60/0x548 [livepatch_nfsd]
>   livepatch: failed to initialize patch 'livepatch_nfsd' for module 'nfsd' (-8)
>   livepatch: patch 'livepatch_nfsd' failed for module 'nfsd', refusing to load module 'nfsd'
> 
> He also proposed three different solutions. We could remove the error
> check in apply_relocate_add() introduced by commit eda9cec4c9a1
> ("x86/module: Detect and skip invalid relocations"). However the check
> is useful for detecting corrupted modules.
> 
> We could also deny the patched modules to be removed. If it proved to be
> a major drawback for users, we could still implement a different
> approach. The solution would also complicate the existing code a lot.
> 
> We thus decided to reverse the relocation patching (clear all relocation
> targets on x86_64). The solution is not
> universal and is too much arch-specific, but it may prove to be simpler
> in the end.
> 
> Reported-by: Josh Poimboeuf <jpoimboe@redhat.com>
> Signed-off-by: Miroslav Benes <mbenes@suse.cz>
> Signed-off-by: Song Liu <song@kernel.org>
> 
> ---
> 
> NOTE: powerpc code has not be tested.
> 

Hi Song,

I just want to provide a quick check in on this patch...

First -- what tree / commit should this be based on?  When I add this
patch on top of a v5.19 based tree, I see:

arch/powerpc/kernel/module_64.c: In function ‘clear_relocate_add’:
arch/powerpc/kernel/module_64.c:781:52: error: incompatible type for argument 1 of ‘instr_is_relative_link_branch’
  781 |                 if (!instr_is_relative_link_branch(*instruction))
      |                                                    ^~~~~~~~~~~~
      |                                                    |
      |                                                    u32 {aka unsigned int}
In file included from arch/powerpc/kernel/module_64.c:20:
./arch/powerpc/include/asm/code-patching.h:122:46: note: expected ‘ppc_inst_t’ but argument is of type ‘u32’ {aka ‘unsigned int’}
  122 | int instr_is_relative_link_branch(ppc_inst_t instr);
      |                                   ~~~~~~~~~~~^~~~~
arch/powerpc/kernel/module_64.c:785:32: error: ‘PPC_INST_NOP’ undeclared (first use in this function); did you mean ‘PPC_INST_COPY’?
  785 |                 *instruction = PPC_INST_NOP;
      |                                ^~~~~~~~~~~~
      |                                PPC_INST_COPY
arch/powerpc/kernel/module_64.c:785:32: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [scripts/Makefile.build:249: arch/powerpc/kernel/module_64.o] Error 1
make[1]: *** [scripts/Makefile.build:466: arch/powerpc/kernel] Error 2
make: *** [Makefile:1849: arch/powerpc] Error 2


Second, I rebased the klp-convert-tree on top of v5.19 here:
https://github.com/joe-lawrence/klp-convert-tree/tree/klp-convert-v7-devel

and I can confirm that at least the x86_64 livepatching selftests
(including the klp-relocation tests added by this tree) do pass.  I
haven't had a chance to try writing new tests to verify this specific
patch, but I'll take a look next week.

Regards,

--
Joe