crypto: Standalone crypto module
Subsystems (11)
v2 cover crypto: Standalone crypto module
This patch series is V2 of the previously submitted standalone crypto kernel module feature patch series.
Major update:
V2 is an improved implementation that reduces 3,000 lines of code (~70%) (shrinking patch numbers from ~160 (V1) down to ~20 (V2)), making the patch set significantly less intrusive to the mainline kernel source tree. (For details on how this reduction is achieved, see Section 4 & 7.) Therefore, compared to the V1 submission where it was organized into 4 patch series, the entire feature is now shipped in one patch series.
Other updates:
1. Update cover letter: Elaborate on how this feature can be used and benefit downstream distros in practice (Section 6). It also clarifies that a stable kernel internal API is not required.
2. Rebase to latest mainline (up to "Merge tag 'v7.1-rc-part1-smbdirect-fixes' of git://git.samba.org/ksmbd").
This feature has been officially launched on the AWS platform, starting from Amazon Linux Kernel 6.18 and later. The full source code is available at https://github.com/amazonlinux/linux/tree/fips-kernel-module.
Details on this feature and what each patch covers can be found below.
## 1. Introduction
Amazon Linux has released a new kernel feature that converts the previously built-in kernel crypto subsystem into a standalone kernel module. This module becomes the carrier of the kernel crypto subsystem and can be loaded at early boot to provide the same functionality as the original built-in crypto. The primary motivation for this modularization is to streamline Federal Information Processing Standards (FIPS) validation, a critical cryptographic certification for cloud computing users doing business with the U.S. government. This feature is publicly available on the AWS platform.
In a bit more detail, previously, FIPS certification was tied to the entire kernel image, meaning non-crypto updates could potentially invalidate certification. With this feature, FIPS certification is tied only to the crypto module. Therefore, once the module is certified, loading this certified module on newer kernels automatically makes those kernels FIPS-certified. As a result, this approach can save re-certification costs and 12-18 months of waiting time by reducing the need for repeated FIPS re-certification cycles.
This document provides technical details on how this feature is designed and implemented for users or developers who are interested in developing upon it, and is organized as follows:
- Section 2 - Getting Started: Quick start on how to enable the feature
- Section 3 - Workflow Overview: Changes this feature brings to build and runtime
- Section 4 - Design Implementation Details: Technical deep-dive into each component
- Section 5 - Customizing and Extending Crypto Module: How to select crypto to be included and extend to new crypto/architectures
- Section 6 - Reusing a Certified Module in Practice: Different reuse and maintenance strategies and their tradeoffs
- Section 7 - Related Work and Comparison
- Section 8 - Summary
## 2. Getting Started
This section provides a quick start guide for developers on how to enable, compile and use the standalone cryptography module feature.
### 2.1 Basic Configuration
The feature is controlled by a single configuration option:
``` CONFIG_CRYPTO_FIPS140_EXTMOD=y ```
What it does: When enabled, automatically redirects a set of cryptographic algorithms from the main kernel into a standalone module crypto/fips140/fips140.ko. The cryptographic algorithms that are redirected need to satisfy all the following conditions, otherwise the cryptography will remain in its original form:
1. Must be configured as built-in (i.e., CONFIG_CRYPTO_*=y). This means cryptography already configured as modular (i.e., CONFIG_CRYPTO_*=m) are not redirected as they are already modularized.
2. Must be among a list, which can be customized by developers as described in Section 5.
When disabled, the kernel behaves as before.
### 2.2 Build Process
Once CONFIG_CRYPTO_FIPS140_EXTMOD=y is set, no additional steps are required. The standalone module will be built automatically as part of the standard kernel build process:
``` make -j$(nproc) # or make vmlinux ```
**What happens automatically (No user action required):**
1. Build the module as crypto/fips140/fips140.ko
2. The cryptography module will be loaded at boot time
3. All kernel cryptographic services will provide the same functionality as before (i.e., prior to introducing this new feature) once boot completes.
### 2.3 Advanced Configuration Options
**Using External Cryptography Module:**
``` CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE=y ```
By default, CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE is not set, meaning the freshly built cryptography module is used. Otherwise, the pre-built standalone cryptography module from fips140_build/crypto/fips140/fips140.ko and modular cryptography such as fips140_build/crypto/aes.ko (need to manually place pre-built modules in these locations before the build) are included in kernel packaging (e.g., during make modules_install) and are used at later boot time.
**Dual Version Support:**
``` CONFIG_CRYPTO_FIPS140_DUAL_VERSION=y ```
Encapsulate two versions of fips140.ko into kernel: one is freshly built for non-FIPS mode usage, another is pre-built specified by fips140_build/crypto/fips140/fips140.ko for FIPS mode usage. The appropriate version is selected and loaded at boot time based on boot time FIPS mode status.
### 2.4 Verification
To verify the feature is working, after install and boot with the new kernel:
``` # Check if fips140.ko module is loaded lsmod | grep fips140 # Check if crypto algorithms are served by the fips140 module cat /proc/crypto | grep module | grep fips140 ```
## 3. Workflow Overview
This section provides an overview without delving into deep technical details of the changes the standalone cryptography module feature introduces. When this feature is enabled, it introduces changes to both the kernel build and booting process.
3.1 Build-Time Changes
Kernel cryptography subsystem consists of both cryptography management infrastructure (e.g., crypto/api.c, crypto/algapi.c, etc), along with hundreds of different cryptography algorithms (e.g., crypto/arc4.c).
**Traditional Build Process:**
Traditionally, cryptography management infrastructure are always built-in to the kernel, while cryptographic algorithms can be configured to be built either as built-in (CONFIG_CRYPTO_*=y) or as separate modular (CONFIG_CRYPTO_*=m) .ko file depending on kernel configuration:
As a result, the builtin cryptography management infrastructure and cryptographic algorithms are statically linked into the kernel binary:
``` cryptographic algorithms source files → compiled as .o objfiles → linked into vmlinux → single kernel binary ``` **With Standalone Cryptography Module:** This feature automatically transforms the builtin cryptographic components into a standalone cryptography module, `fips140.ko`. To do so, it develops a new kernel build rule `crypto-objs-$(CONFIG_CRYPTO_*)` such that, once this build rule is applied to a cryptographic algorithm, such cryptographic algorithm will be automatically collected into the cryptography module if it is configured as built-in (i.e, `CONFIG_CRYPTO_*=y`), for example: ``` // in crypto/asymmetric_keys/Makefile - obj-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o + crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o ```
Such build change allows the modularization transformation to only affect selected cryptographic algorithms (i.e, where the crypto-objs-$(CONFIG_CRYPTO_*) is applied).
Then, after the fips140.ko is generated, it will be embedded back into main kernel vmlinux as a replacement part. The purpose of this embedding, instead of traditionally putting the fips140.ko into filesystem, is a preparation to allow the module to be loaded early enough even before the filesystem is ready.
The new build process is illustrated below.
``` cryptographic algorithms source files → compiled as .o objfiles → automatically collected and linked into fips140.ko → embedded fips140.ko into vmlinux as a replaceable binary ```
### 3.2 Runtime Changes
**Traditional Boot Process:**
The kernel initializes the cryptographic subsystem early during boot, executing each cryptographic initialization routine accordingly. These initialization routines may depend on other cryptographic components or other kernel subsystems, so their invocation follows a well-defined execution order to ensure they are initialized before their first use.
``` kernel starts → cryptography subsystem initialization → cryptography subsystem available → other components use cryptography ```
**With Standalone Cryptography Module:**
At the start of kernel boot, compared to a regular kernel, the first major change introduced by this feature is that no cryptography services are initially available — since the entire cryptography subsystem has been decoupled from the main kernel. To ensure that the cryptography subsystem becomes available early enough (before the first kernel component that requires cryptography services), the standalone cryptography kernel module must be loaded at a very early stage, even before the filesystem becomes available.
However, the regular module loading mechanism relies on placing kernel modules in the filesystem and loading them from there, which creates a chicken-and-egg problem — the cryptography module cannot be loaded until the filesystem is ready, yet some kernel components may require cryptography services even before that point.
To address this, the second change introduced by this feature is that the cryptography kernel module is loaded directly from memory, leveraging the earlier compilation changes that embed the module binary into the main kernel image. Afterward, the feature includes a “plug-in” mechanism that connects the decoupled cryptography subsystem back to the main kernel, ensuring that kernel cryptography users can correctly locate and invoke the cryptography routine entry points.
Finally, to ensure proper initialization, the feature guarantees that all cryptography algorithms and the cryptography management infra execute their initialization routines in the exact same order as they would if they were built-in.
The process described above is illustrated below.
``` kernel starts → no cryptography available → load fips140.ko from memory → plug cryptography back to kernel → module initialization → cryptographic services available → other components use cryptography ```
## 4. Design Implementation Details
While the earlier sections provide a holistic view of how this feature shapes the kernel, this section provides deeper design details on how these functionalities are realized. There are three key design components:
1. A specialized compile rule that automatically compiles and collects all built-in cryptographic algorithm object files to generate the final module binary under arbitrary kernel configurations, and then embeds the generated binary into the main kernel image for early loading.
2. A mechanism to convert interactions between the cryptography subsystem and the main kernel into a pluggable interface.
3. A module loading and initialization process that ensures the cryptography subsystem is properly initialized as if it were built-in.
### 4.1. Specialized Compilation System
**Automatic Collection and Linking of Built-in Cryptographic Algorithm Objects:**
The first step in generating the fips140.ko module is to compile and collect built-in cryptographic components (i.e., those specified by CONFIG_CRYPTO_*=y).
Traditionally, the existing module build process requires all module components (e.g., source files) to reside in a single directory. However, this approach is not suitable for our case, where hundreds of cryptographic algorithm source files are scattered across multiple directories.
A naïve approach would be to create a separate Makefile that duplicates the original build rules with adjusted paths.
However, this method is not scalable due to the large number of cryptographic build rules, many of which are highly customized and can vary under different Kconfig settings, making such a separate Makefile even more complex. Moreover, this approach cannot ensure that built-in cryptographic algorithms are completely removed from the main kernel, which would result in redundant cryptographic code being included in both the kernel and the module.
To tackle this challenge, we automated the object collection and linking process by introducing special build logic for the kernel cryptography subsystem. Specifically, to automatically collect cryptography object files while preserving their original compilation settings (such as flags, headers, and paths), we introduced a new compilation rule:
```
crypto-objs-y += *.o
```
This replaces the original `obj-y += *.o` rule in cryptography Makefiles later, for example:
```
// in crypto/asymmetric_keys/Makefile
- obj-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o
+ crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o
asymmetric_keys-y := \
asymmetric_type.o \
restrict.o \
signature.o
```in the cryptography subsystem Makefiles, allowing most of the existing Makefile logic to be reused.
As a result, when the standalone cryptography module feature is enabled, any cryptographic algorithm configured as built-in (for example, crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o where CONFIG_ASYMMETRIC_KEY_TYPE=y) will be automatically collected and linked into a single final object binary, fips140.o. During this process, a special compilation flag (-DFIPS_MODULE=1) is applied to instruct each object file to be compiled in a module-specific manner. This flag will later be used to generate the pluggable interface on both the main kernel side and the module side from the same source code.
The implementation details are as follows: it follows a similar methodology used by the obj-y collection process for building vmlinux.o. The crypto-objs-y rule is placed in scripts/Makefile.build, which is executed by each directory Makefile to collect the corresponding crypto object files. Each directory then creates a crypto-module.a archive that contains all crypto-objs-y += <object>.o files under that directory. In the parent directories, these crypto-module.a archives are recursively included into the parent’s own crypto-module.a, and this process continues upward until the final fips140.o is generated.
**A Separate Module Generation Pipeline for Building the Final Kernel Module from Linked Cryptographic Algorithm Object:** With the linked cryptographic algorithm object (i.e., fips140.o), the next step is to generate the final kernel module, fips140.ko.
A direct approach would be to inject the fips140.ko module build into the existing modules generation pipeline (i.e., make modules) by providing our pre-generated fips140.o. However, we choose not to do this because it would create a circular make rule dependency (which is invalid in Makefiles and causes build failures), resulting in mutual dependencies between the modules and vmlinux targets (i.e., modules:vmlinux and vmlinux:modules at the same time). This happens for the following reasons:
1. Since we will later embed fips140.ko into the final kernel image (as described in the next section), we must make vmlinux depend on fips140.ko. In other words: vmlinux: fips140.ko.
2. When the kernel is built with CONFIG_DEBUG_INFO_BTF_MODULES=y, it requires: modules: vmlinux. This is because CONFIG_DEBUG_INFO_BTF_MODULES=y takes vmlinux as input to generate BTF info for the module, and inserts such info into the .ko module by default.
3. If we choose to inject fips140.ko into make modules, this would create a make rule dependency: fips140.ko: modules. Combined with items 1 and 2, this eventually creates an invalid circular dependency between vmlinux and modules.
Due to these reasons, the design choice is to use a separate make pipeline (defined as fips140-ready in the Makefile). This new pipeline reuses the same module generation scripts used by make modules but adds additional logic in scripts/Makefile.{modfinal|modinst|modpost} and scripts/mod/modpost.c to handle module symbol generation and verification correctly.
**A Seamless Process That Embeds the Generated Binary Into the Main Kernel Image for Early Loading:**
As mentioned earlier, in order to load the standalone cryptography module early in the boot process—before the filesystem is ready—the module binary must be embedded into the final kernel image (i.e., vmlinux) so that it can be loaded directly from memory. We intend for this embedding process to be completely seamless and automatically triggered whenever vmlinux is built (i.e., during make vmlinux).
To achieve this, the feature adds a Make dependency rule so that vmlinux depends on fips140.ko.
It also modifies the vmlinux link rules (i.e., arch/<arch>/kernel/vmlinux.lds.S, scripts/Makefile.vmlinux, and scripts/link-vmlinux.sh) so that the generated module binary is finally combined with vmlinux.o.
In addition, we allow multiple cryptography module binary versions (for example, a certified cryptography binary and a latest, up-to-date but uncertified one) to be embedded into the main kernel image to serve different user needs. This design allows regular (non-FIPS) users to benefit from the latest cryptographic updates, while FIPS-mode users continue to use the certified cryptography module.
To support this, we introduce an optional configuration, CONFIG_CRYPTO_FIPS140_DUAL_VERSION. When enabled, this option allows two cryptography module versions to be embedded within a single kernel build and ensures that the appropriate module is selected and loaded at boot time based on the system’s FIPS mode status.
### 4.2. Pluggable Interface Between the Built-in Cryptography Subsystem and the Main Kernel
Although the module binary (fips140.ko) has been embedded into the final kernel image (vmlinux) as described in the previous section, it is not linked to the kernel in any way. This is because fips140.ko is embedded in a data-only manner, so the main kernel cannot directly call any functions or access any data defined in the module binary. Since the main kernel and modules can only interact through exported symbols (i.e., via EXPORT_SYMBOL()), this also applies to the crypto kernel module — the main kernel can only interact with the crypto functions and variables defined in the crypto module through exported symbols, meaning these functions and variables must also have their symbols exported in the module after they are moved from the main kernel to the module.
However, simply making these crypto symbols symbol-exported in the module without additional handling would cause the kernel to fail to compile. This is because the existing kernel module symbol resolution mechanism is essentially one-way: it supports symbols defined in the main kernel and referenced by kernel modules. However, it does not support the reverse case — symbols defined in a kernel module but used by the main kernel — which is exactly the crypto module case, as there are many crypto users still residing in the main kernel. The reason is that compilation of the main kernel requires all symbol addresses to be known to achieve a successful linking phase.
To address this, we introduce a pluggable interface to support this reverse-direction symbol resolution between crypto symbols defined in the module and referenced by the main kernel, by placing **address placeholders** at all crypto usage points in the main kernel. These address placeholders are initially set to NULL during compilation to provide a concrete address that satisfies the linking phase. Then, during runtime, once the cryptography kernel module is loaded, these placeholders are updated to the correct addresses before their first use in the main kernel. In the rest of this section, we first introduce this pluggable interface mechanism, and then explain how to apply it to the built-in cryptographic algorithms and variables.
**The Pluggable Interface Mechanism:**
There are two types of address holders used to achieve this pluggable interface:
- Function addresses (the majority): We use a trampoline to redirect the original jump instruction to another location whose target destination is held by the value of a function pointer. To avoid additional security concerns, such as the function pointer being arbitrarily modified, these function pointers are made __ro_after_init to ensure they cannot be modified after kernel init. We implement this function-address placeholder as the DEFINE_CRYPTO_FN_REDIRECT() wrapper.
- Variable addresses (the remaining smaller portion): For these, we use a pointer of the corresponding data type. We implement this address placeholder as the DECLARE_CRYPTO_VAR() and DEFINE_CRYPTO_API_STUB() wrappers:
These wrappers are applied to each symbol-exported (i.e., EXPORT_SYMBOL()) cryptographic function and variable (details on how to apply them are described later). Once applied, the wrappers are compiled differently for the main kernel and for the built-in cryptographic algorithm source code—acting as the “outlet” and the “plug,” respectively—using different compilation flags (-DFIPS_MODULE) introduced by our customized build rules described earlier.
As a result, the kernel can successfully compile even when the built-in cryptographic algorithms are removed, thanks to these address placeholders. At boot time, the placeholders initially hold NULL, but since no cryptography users exist at that stage, the kernel can still start booting correctly. After the cryptography module is loaded, the placeholders are dynamically updated to the correct addresses later (by do_crypto_var() and do_crypto_fn(), described in a later section).
**Applying the Pluggable Interface Mechanism to Cryptographic Algorithms:**
To apply these pluggable interface wrappers to a cryptographic algorithm and make them take effect, we follow the steps below (using crypto/asymmetric_keys/asymmetric_type.c as an example):
1. **Apply crypto-objs-y compile rule to the cryptographic algorithm:**
```
// in crypto/asymmetric_keys/Makefile
- obj-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o
+ crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o
asymmetric_keys-y := \
asymmetric_type.o \
restrict.o \
signature.o
```2. **Locate the communication point between the cryptographic algorithm and the main kernel:**
The cryptography subsystem is designed such that most interactions between the main kernel and cryptographic algorithms occur through exported symbols using EXPORT_SYMBOL() wrappers. This kernel design exists because most cryptographic algorithm implementations must support both built-in and modular modes.
Consequently, the cryptographic functions and variables exported by EXPORT_SYMBOL() are a well-defined and identifiable interface between the cryptography subsystem and the main kernel:
```
// in crypto/asymmetric_keys/asymmetric_type.c
//Exported cryptographic function:
bool asymmetric_key_id_same(const struct asymmetric_key_id *kid1,
const struct asymmetric_key_id *kid2) {...}
EXPORT_SYMBOL_GPL(asymmetric_key_id_same);
//Exported cryptographic variable:
struct key_type key_type_asymmetric = {...};
EXPORT_SYMBOL_GPL(key_type_asymmetric);
```3. **Redirect crypto symbol references in the main kernel to address placeholders:**
With the placeholders in place, the remaining problem is directing the main kernel to use them rather than the original symbols. Since all crypto users must include the corresponding header files to obtain function and variable declarations, the headers are a natural place to perform this redirection. Each declaration is transformed using a macro that hooks it to the corresponding placeholder.
For exported variable symbols (a small number, ~10 symbols), their declaration in the header file is replaced with the DECLARE_CRYPTO_VAR() wrapper to redirect variable access from a concrete address to a placeholder:
``` // in include/keys/asymmetric-type.h // for exported cryptographic variables: - struct key_type key_type_asymmetric; + DECLARE_CRYPTO_VAR(CONFIG_ASYMMETRIC_KEY_TYPE, key_type_asymmetric, struct key_type, ); + #if defined(CONFIG_CRYPTO_FIPS140_EXTMOD) && !defined(FIPS_MODULE) && IS_BUILTIN(CONFIG_ASYMMETRIC_KEY_TYPE) + #define key_type_asymmetric (*((struct key_type*)CRYPTO_VAR_NAME(key_type_asymmetric))) + #endif ```
By doing so, we can automatically force all cryptography users to go through the placeholders, because those users already include the same header file.
The wrapper also takes the cryptographic algorithm Kconfig symbol as a parameter, so that when a cryptographic algorithm is built as a module (for example, CONFIG_ASYMMETRIC_KEY_TYPE=m), the original function declarations remain unchanged and are not affected.
For exported function symbols (the majority, ~hundreds), a similar approach could be taken, but instead we use an automated method to redirect function address usage to placeholders during the kernel compilation process. This makes the crypto module implementation less intrusive to the kernel source tree, as no header file modifications are needed. To achieve this, a linker option --wrap=<symbols-to-redirect> is leveraged to rename all uses of crypto functions in the main kernel to dedicated trampolines that act as address placeholders. As a consequence, all references to crypto function symbols are automatically redirected to the address placeholders, avoiding mass intrusive changes to the mainline kernel source tree.
4. **Add the address-placeholder definition wrappers into a dedicated file fips140-var-redirect.c:**
After redirecting crypto users to use address placeholders, we also need to add the definitions of those address placeholders.
For exported variable symbols (a small number, ~10 symbols), add the placeholder definition wrapper DEFINE_CRYPTO_VAR_STUB to a dedicated file fips140-var-redirect.c.
``` // in crypto/fips140/fips140-var-redirect.c // for exported cryptographic variables: + #undef key_type_asymmetric + DEFINE_CRYPTO_VAR_STUB(key_type_asymmetric); + #endif ```
This file will be compiled separately and acts as both the “outlet” and the “plug” for the main kernel and the cryptography module, respectively.
For exported function symbols (the majority, ~hundreds), a similar wrapper DEFINE_CRYPTO_FN_REDIRECT() is used, but again, the application of this wrapper is automated, so there is no need to manually apply it. Instead, it is generated automatically by the script crypto/fips140/gen-fips140-fn-redirect.sh on every kernel build.
We apply the above steps to both architecture-independent and architecture-specific cryptographic algorithms.
### 4.3. Initialization Synchronization
To ensure the embedded fips140.ko module binary provides the same cryptography functionality as the regular kernel, the kernel needs:
1. A module loader to load the module binary directly from memory,
2. A mechanism to plug the module back into the kernel by updating the address placeholders, and
3. Correct cryptography subsystem initialization, as if the cryptographic algorithms were still built-in.
**Directly Load Module Binary from Memory:**
Regular modules are loaded from the filesystem and undergo signature verification on the module binary, which relies on cryptographic operations. However, since we have already fully decoupled the cryptography subsystem, we must skip this step for this fips140.ko module. To achieve this, we add a new loader function load_crypto_module_mem() that can load the module binary directly from memory at the designed address without checking the signature. Since the module binary is embedded into main kernel in an ELF section, as specified in the linker script:
```
// in arch/<arch>/kernel/vmlinux.lds.S
.fips140_embedded : AT(ADDR(.fips140_embedded) - LOAD_OFFSET) {
. = ALIGN(8);
_binary_fips140_ko_start = .;
KEEP(*(.fips140_module_data))
_binary_fips140_ko_end = .;
}
```Therefore, the runtime memory address of the module can be accessed directly by the module loader to invoke the new loader function load_crypto_module_mem().
**Plug Back the Module by Updating Address Placeholder Values:**
To update the address placeholders in the main kernel to the correct addresses matching the loaded module, after compilation the placeholder values are placed into dedicated key-value data structures, which reside in ELF sections __crypto_fn_keys and __crypto_var_keys. This can be seen from the definition of the placeholder's key-value data structure:
```
#define __CRYPTO_FN_KEY(sym) \
extern void *__fips140_fn_ptr_##sym; \
static struct _crypto_fn_key __##sym##_fn_key \
__used \
__section("__crypto_fn_keys") \ // Place in a dedicated ELF Section
__aligned(__alignof__(struct _crypto_fn_key)) = { \
.ptr = (void **)&__fips140_fn_ptr_##sym, \
.func = (void *)&sym, \
};
#define DEFINE_CRYPTO_VAR_STUB(name) \
static struct crypto_var_key __crypto_##name##_var_key \
__used \
__section("__crypto_var_keys") \ // Place in a dedicated ELF Section
__aligned(__alignof__(struct crypto_var_key)) = \
{ \
.ptr = &CRYPTO_VAR_NAME(name), \
.var = (void*)&name, \
};
```The purpose of doing this is to allow the main kernel to quickly locate the placeholders and update them to the correct addresses. The update functions are defined as do_crypto_var() and do_crypto_fn(), which are executed at module load.
As a result, all cryptography users in the main kernel can now call the cryptographic functions as if they were built-in.
**Initialize Cryptography Subsystem as if it Were Built-in:**
Cryptographic components must be properly initialized before use, and this initialization is typically achieved through dedicated initialization functions (e.g., module_init(crypto_init_func) or late_initcall(crypto_init_func)). Traditionally, these init functions are executed automatically as part of the kernel boot phase. However, now that they are moved to a crypto module, there needs to be a way to collect and execute them.
To collect these init functions, the init wrappers (e.g., module_init() and late_initcall) are modified to automatically place the wrapped crypto init function into a dedicated list in the crypto module, for example:
```
// in include/linux/module.h
#define subsys_initcall(fn) \
static initcall_t __used __section(".fips_initcall0") \ // a dedicated list
__fips_##fn = fn;
#define module_init(initfn) \
static initcall_t __used __section(".fips_initcall1") \ // a dedicated list
__fips_##initfn = initfn;
```By doing so, all init functions are now aggregated for execution by run_initcalls() at module load.
Besides collecting these crypto init functions, rather than simply executing them, another key consideration is that their execution order often has strict requirements. In other words, for these collected crypto init functions, we must ensure that their initialization order is preserved as before because failure to follow the correct order can result in kernel panic.
To address this, we introduce a synchronization mechanism between the main kernel and the module to ensure all cryptographic algorithms are executed in the correct kernel boot phase. In more details, we spawn the module initialization process fips_loader_init() as an async thread fips140_sync_thread(), in which we call run_initcalls() to execute the initialization calls of each cryptographic algorithm. Then, we introduce synchronization helpers such as wait_until_fips140_level_sync(int level) to ensure the initialization order of all cryptographic algorithms is synchronized with the main kernel.
## 5. Customization and Extension of Cryptography Module
This section describes how developers can customize which cryptographic algorithms are included in the standalone cryptography module, as well as extend this feature to other cryptographic algorithms or hardware architectures.
### 5.1. Cryptography Selection Mechanism
The feature automatically includes cryptographic algorithms that meet specific criteria:
1. **Built-in Configuration**: Only cryptographic algorithms configured as CONFIG_CRYPTO_*=y are candidates for inclusion
2. **Explicit Inclusion**: Cryptographic algorithms must be explicitly converted using the crypto-objs-$(CONFIG__CRYPTO_*) build rule
### 5.2. Extend Support to New Cryptographic Algorithms
To extend support to a new cryptographic algorithm in the standalone module, follow these steps:
**Step 1: Update the Makefile**
``` # in crypto/[algorithm]/Makefile - obj-$(CONFIG_CRYPTO_ALGORITHM) += algorithm.o + crypto-objs-$(CONFIG_CRYPTO_ALGORITHM) += algorithm.o ```
For Architecture-Specific Cryptographic Algorithms:
- Apply the crypto-objs- rule in the appropriate arch/*/crypto/Makefile
**Step 2: Add Pluggable Interface Support**
If the cryptographic algorithm has symbol-exported variables via EXPORT_SYMBOL(), add the pluggable interface wrappers. There is no need to manually apply wrappers for symbol-exported functions:
``` // Example: in include/keys/asymmetric-type.h // for exported cryptographic variables: - struct key_type key_type_asymmetric; + DECLARE_CRYPTO_VAR(CONFIG_ASYMMETRIC_KEY_TYPE, key_type_asymmetric, struct key_type, ); + #if defined(CONFIG_CRYPTO_FIPS140_EXTMOD) && !defined(FIPS_MODULE) && IS_BUILTIN(CONFIG_ASYMMETRIC_KEY_TYPE) + #define key_type_asymmetric (*((struct key_type*)CRYPTO_VAR_NAME(key_type_asymmetric))) + #endif ``` Then, add the corresponding stubs in `crypto/fips140/fips140-var-redirect.c`: ``` + #undef key_type_asymmetric + DEFINE_CRYPTO_VAR_STUB(key_type_asymmetric); + #endif ```
For Architecture-Specific Cryptographic Algorithms:
- Include architecture-specific stubs in arch/*/crypto/fips140/fips140-var-redirect.c:
### 5.3. Architecture-Specific Extensions
**Extending to New Architectures:**
Currently supported architectures are x86_64 and ARM64. To extend this feature to additional architectures:
1. **Update Linker Scripts**: Add ELF sections in arch/[new-arch]/kernel/vmlinux.lds.S:
```
.fips140_embedded : AT(ADDR(.fips140_embedded) - LOAD_OFFSET) {
. = ALIGN(8);
_binary_fips140_ko_start = .;
KEEP(*(.fips140_module_data))
_binary_fips140_ko_end = .;
}
```2. **Create Architecture-Specific Files**: Set up arch/[new-arch]/crypto/fips140/ directory with Makefile and fips140-var-redirect.c following the pattern used in x86_64 and ARM64.
## 6. Reusing a certified module in practice
With the crypto subsystem restructured as a loadable module, a previously certified module can be reused across kernel updates. Crypto development does not freeze in the meantime — updated modules can always be submitted for fresh certification. The reuse of an already-certified module is intended to simply bridge the gap until the new certification arrives, that is, using the old one before the new one gets certified. How much of the certified module to reuse, however, involves a tradeoff between crypto feature availability, certification turnaround time, and engineering effort.
The most conservative option is no reuse at all: abandon the previous certification and submit the updated crypto module for a full FIPS certification. This allows the crypto subsystem to benefit from the latest upstream changes, but at the cost of the full 12-to-18-month waiting period, according to experiences from downstream distributions.
At the opposite end, distributions can choose to reuse the certified module binary entirely on a newer kernel. This enables the new kernel to be validated on day one. The tradeoff is clear: besides forgoing crypto updates, engineers must ensure ABI compatibility between the updated kernel and the module.
A middle ground is to reuse only the source code of the certified module, freezing it and recompiling against the updated main kernel. Since the source code remains unchanged, the new module can go through a Non-Security Relevant ([NSRL](https://csrc.nist.gov/csrc/media/Projects/cryptographic-module-validation-program/documents/fips%20140-3/FIPS-140-3-CMVP%20Management%20Manual.pdf)) process — a simpler FIPS re-certification that typically reduces the waiting time from 12–18 months down to 3–4 months. Compared to reusing the binary entirely, this option requires less engineering effort, since engineers need only maintain source-level API compatibility (i.e., by patching the main kernel source code, not touching the crypto module source code) rather than binary-level ABI compatibility between the crypto module and the main kernel.
In summary, converting the kernel crypto subsystem into a loadable module enables reuse of a certified module across kernel updates. Whether through binary reuse, source-code reuse, or fresh certification, different choices represent different tradeoffs, and distributions can balance crypto feature availability, certification turnaround time, and engineering effort according to their needs.
## 7. Related Work and Comparison
The idea of modularizing kernel cryptographic functionality has also attracted attention from other Linux distributions as well as Linux-kernel-based platforms that are not traditional distributions. Specifically, there are two related efforts: one from [Android's GKI kernel](https://source.android.com/docs/core/architecture/kernel/gki-fips140-module) and another from [Oracle Linux](https://git.kernel.org/pub/scm/linux/kernel/git/vegard/linux-fips140.git/log/?h=fips140). While Amazon Linux incorporated several valuable ideas from these efforts (and have acknowledged them in the patch commits—thank you again!), this section highlights the key differences between those approaches and this approach. The goal is to describe the trade-offs and design choices objectively, rather than to criticize other implementations.
### 7.1. Comparison with Android's GKI
Android's work is the earliest one on modularizing kernel cryptographic code, and it targets a non-intrusive approach to the core GKI kernel, with the goal of minimizing modifications to the kernel source. To achieve this, the crypto module relies on several interception or "hijacking" techniques that intervene in the core kernel execution path.
While this approach minimizes kernel code changes, we don't adopt such an approach for several reasons. First, these interception mechanisms tightly depend on internal kernel crypto subsystem behavior, making them fragile across major kernel updates thus less suitable to reuse the same module on newer major kernel versions. Second, this design requires substantial additional cryptographic code duplication, which impacts maintainability. Finally, the solution only supports a fixed set of cryptographic algorithms, making it non-general and difficult to extend.
In contrast, our design integrates directly into the Linux kernel source tree, avoids duplicated cryptographic implementations, supports arbitrary kernel configuration settings, and works with any chosen set of cryptographic algorithms.
### 7.2. Comparison with Oracle Linux
Oracle’s work was developed concurrently with this approach. The primary differences between Oracle’s approach and Amazon's lie in build integration, pluggable interface design, and module initialization.
**Build Integration:**
Oracle's module is implemented as an out-of-tree module with a separate Makefile. This introduces three major reasons we don't adopt such an approach:
*First*, the separate Makefile duplicates many kernel build rules, which increases maintenance cost, as upstream kernel build changes must be tracked and replicated. One concrete example can be seen below:
in Oracle's module makefile
``` fips140-y += crypto/skcipher.o fips140-y += crypto/lskcipher.o ``` However, in upstream, the corresponding build logic is more complex and configuration-dependent: ``` crypto_skcipher-y += lskcipher.o crypto_skcipher-y += skcipher.o obj-$(CONFIG_CRYPTO_SKCIPHER2) += crypto_skcipher.o ifeq ($(CONFIG_BPF_SYSCALL),y) obj-$(CONFIG_CRYPTO_SKCIPHER2) += bpf_crypto_skcipher.o endif ``` As shown above, when `CONFIG_BPF_SYSCALL` is enabled, `bpf_crypto_skcipher.o` must also be included. Tracking such dependencies is hard in the duplicated Makefile approach. In contrast, our approach integrates seamlessly into the kernel build system by introducing a customized build rule (`crypto-objs-*`) rather than relying on a duplicated Makefile, such that this is handled correctly by reusing the existing kernel build logic: ``` crypto_skcipher-y += lskcipher.o crypto_skcipher-y += skcipher.o - obj-$(CONFIG_CRYPTO_SKCIPHER2) += crypto_skcipher.o + crypto-objs-$(CONFIG_CRYPTO_SKCIPHER2) += crypto_skcipher.o ifeq ($(CONFIG_BPF_SYSCALL),y) - obj-$(CONFIG_CRYPTO_SKCIPHER2) += bpf_crypto_skcipher.o + crypto-objs-$(CONFIG_CRYPTO_SKCIPHER2) += bpf_crypto_skcipher.o endif ```
As a result, such a Makefile-duplication approach does not scale well across all kernel configurations and does not easily support arbitrary sets of cryptographic algorithms.
*Second*, since the module is to be embedded as part of the kernel image (i.e., vmlinux) as described earlier, the module build must be triggered automatically as part of the vmlinux build process to achieve a seamless build workflow. However, Oracle's module build is not tightly integrated into the kernel build framework and requires special build commands (e.g., first do make M=fips140/ specifically, then do some shell command and finally make).
In contrast, our approach improves this aspect by integrating the module build tightly into the regular kernel build, so the build process is seamless and automatic with regular build and packaging processes such as make or make vmlinux or make install.
**Pluggable Interface:**
There are several differences in the pluggable interface design.
*First*, the approach we use avoids massive intrusion to the main kernel source tree (i.e., significantly fewer lines of source code modification), making it easier to maintain and merge to mainline. This reduction is mainly achieved by automating the conversion process of crypto function symbols to pluggable symbols. Specifically, the existing approach requires modifications to the cryptographic implementation (.c) files and the declaration (.h) header files of crypto function symbols, as well as the code pattern of crypto's initialization routine (e.g., replace module_init(init_fn) with crypto_module_init(init_fn)). In contrast, our approach fully automates this process without requiring any source code modification. As a result, our method achieves a reduction of ~3K lines of code (~70% of total), making it easier to maintain and merge to mainline.
*Second*, we avoid duplicate crypto code so only keep one crypto code in kernel memory, while existing work keeps two crypto code even if these crypto code are from the same source code. This is due to the way Oracle defines pluggable interface macros in crypto/api, where its design requires some cryptographic code to remain compiled into the main kernel image in addition to the code inside the standalone cryptography module. Keeping two crypto code is ok if these code are different and designed to be used for different runtime modes (i.e., FIPS/non-FIPS mode), but will be unnecessary if both crypto code are the same.
In contrast, the approach we use can flexibly support both choices: keep one cryptography subsystem, or two different crypto subsystems. To do so, we introduce an option CRYPTO_FIPS140_DUAL_VERSION such that when it is disabled, we only keep one cryptographic subsystem in the cryptography module while completely removing it from the main kernel; and when it is enabled, we allow having two different modules carrying different cryptography for different kernel runtime modes (i.e., FIPS and non-FIPS mode).
*Third*, prior approaches mainly support making cryptographic function calls pluggable, while our approach extends pluggability to cryptographic variables as well.
*Fourth*, prior approach requires all cryptographics that we care (for any purpose such as those within FIPS boundary) to be included within a single kernel module fips140.ko (e.g., when CONFIG_CRYPTO_AES=m, it cannot be aes.ko but must be within fips140.ko). However, this requirement limits the inherent benefit of a kernel module (i.e., on-demand loading for memory efficiency). In contrast, our approach allows the cryptographic we care remain its original modular if it is configured as being so (i.e., if CONFIG_CRYPTO_AES=m, the aes will still be as aes.ko but not forced to fips140.ko) up to the .config setting. One benefit of this design is that it does not impose strict requirements on .config setting (i.e., a cryptography .config can be set to both =y|m while existing work must be set as =y), preserving configuration flexibility.
To support so, for any cryptography within the interest (i.e., whose makerule has been replaced with crypto-objs-*) but configured as build-as-module (i.e., CONFIG_CRYPTO_*=m), its compiled .ko binary will be marked automatically, such that the loader will have a way to recognize to perform some interest-specific processing (e.g., registered as FIPS-required flag) if needed. And the pluggable interface can also adjust its coverage automatically based on different CONFIG_CRYPTO_*=y|=m settings. This is achieved by letting the pluggable interface macro to take CONFIG_CRYPTO_* option as a parameter to recognize the .config setting.
**Module Initialization:**
Oracle's initialization routine does not guarantee preservation of the original crypto initialization order (i.e., the order they should follow if they were originally built-in in the main kernel), which limits its ability to support arbitrary combinations of cryptographic algorithms. This is because the crypto initialization routine in the module is executed too early, such that all module crypto is initialized before the cryptography init in the main kernel. So if there is a crypto in the module (e.g., a crypto init defined as late_init()) that depends on a cryptography (whose init is defined as module_init()) in the main kernel, since the one in the main kernel should be executed earlier (but because the module init is too early, it makes the crypto in the main kernel executed too late), such a case will break the kernel boot process.
Our design, on the other hand, introduces explicit initialization synchronization mechanisms between cryptography's init routine in the module and in the main kernel that can preserve the original built-in initialization order. As a result, our module supports any chosen crypto set to be included in the module.
### 7.3. Comparison Summary
Overall, combined with differences in coding style and integration strategy, the proposed approach has less source-code modification, is more seamlessly integrated with the upstream Linux kernel, making it more generalizable across different kernel configuration settings, and the changed behavior more invisible to kernel users.
## 8. Summary
In this patch series, Amazon Linux proposes a new kernel feature that decouples the built-in crypto subsystem into a dedicated kernel module. To achieve this, several key mechanisms are designed, including specialized compile rules, a novel pluggable interface mechanism, and a module-loading initialization process. This feature is designed in an upstream-friendly manner so that it can support arbitrary kernel configuration settings and arbitrary chosen sets of cryptographic algorithms. It has been officially launched with the Amazon Linux Kernel 6.18 as well as future kernels.
---
Written by Jay Wang <[email protected]> <[email protected]>, Amazon Linux
Jay Wang (17):
crypto: add Kconfig options for standalone crypto module
crypto: add module entry for standalone crypto kernel module
build: special compilation rule for building the standalone crypto module
build: Add ELF marker for crypto-objs-m modules
crypto: add pluggable interface for module symbols referenced by the main kernel
crypto: dedicated ELF sections for collected crypto initcalls
crypto: fips140: add crypto module loader
build: embed the standalone crypto module into vmlinux
module: skip modversion checks for crypto modules
build: add CONFIG_DEBUG_INFO_BTF_MODULES support for the standalone crypto kernel module Allow selective crypto module loading at boot based on FIPS mode
Execute crypto initcalls during module initialization
crypto: fips140: add module integrity self-check
crypto: convert exported symbols in architecture-independent crypto to pluggable symbols x86/crypto: convert exported symbols in x86 crypto to pluggable symbols arm64/crypto: convert exported symbols in arm64 crypto to pluggable symbols Add standalone crypto kernel module technical documentation
Vegard Nossum (2):
module: allow kernel module loading directly from memory crypto/algapi.c: skip crypto_check_module_sig() for the standalone crypto module
Makefile | 106 +++++- arch/arm64/crypto/Makefile | 23 +- arch/arm64/crypto/fips140/Makefile | 14 + arch/arm64/crypto/fips140/fips140-var-redirect.c | 0 arch/arm64/kernel/vmlinux.lds.S | 40 +++ arch/x86/crypto/Makefile | 41 +-- arch/x86/crypto/fips140/Makefile | 14 + arch/x86/crypto/fips140/fips140-var-redirect.c | 0 arch/x86/kernel/vmlinux.lds.S | 40 +++ certs/system_keyring.c | 1 + crypto/Kconfig | 1 + crypto/Makefile | 207 ++++++------ crypto/algapi.c | 14 +- crypto/asymmetric_keys/Makefile | 16 +- crypto/asymmetric_keys/restrict.c | 2 + crypto/asymmetric_keys/verify_pefile.c | 1 + crypto/async_tx/Makefile | 12 +- crypto/fips140/Kconfig | 70 ++++ crypto/fips140/Makefile | 25 ++ crypto/fips140/README | 404 +++++++++++++++++++++++ crypto/fips140/fips140-crypto-module-marker.h | 8 + crypto/fips140/fips140-fn-redirect.c | 10 + crypto/fips140/fips140-loader.c | 195 +++++++++++ crypto/fips140/fips140-module.c | 155 +++++++++ crypto/fips140/fips140-module.h | 30 ++ crypto/fips140/fips140-var-redirect.c | 78 +++++ crypto/fips140/fips140.lds | 38 +++ crypto/fips140/gen-fips140-fn-redirect.sh | 28 ++ crypto/krb5/Makefile | 2 +- include/asm-generic/vmlinux.lds.h | 2 + include/crypto/cast_common.h | 17 +- include/crypto/fips140-fn-redirect.h | 63 ++++ include/crypto/fips140-redirect.h | 92 ++++++ include/crypto/md5.h | 8 +- include/crypto/public_key.h | 7 +- include/crypto/rng.h | 1 + include/crypto/sm4.h | 13 +- include/keys/asymmetric-type.h | 7 +- include/linux/export.h | 22 ++ include/linux/init.h | 10 + include/linux/module.h | 25 ++ include/uapi/linux/module.h | 6 + init/main.c | 4 + kernel/bpf/btf.c | 20 ++ kernel/module/internal.h | 1 + kernel/module/main.c | 185 +++++++++-- kernel/module/version.c | 9 + kernel/params.c | 3 +- scripts/Makefile.build | 124 ++++++- scripts/Makefile.modfinal | 38 +++ scripts/Makefile.modinst | 13 +- scripts/Makefile.modpost | 25 ++ scripts/Makefile.vmlinux | 59 +++- scripts/link-vmlinux.sh | 26 ++ scripts/mod/modpost.c | 24 +- 55 files changed, 2182 insertions(+), 197 deletions(-) create mode 100644 arch/arm64/crypto/fips140/Makefile create mode 100644 arch/arm64/crypto/fips140/fips140-var-redirect.c create mode 100644 arch/x86/crypto/fips140/Makefile create mode 100644 arch/x86/crypto/fips140/fips140-var-redirect.c create mode 100644 crypto/fips140/Kconfig create mode 100644 crypto/fips140/Makefile create mode 100644 crypto/fips140/README create mode 100644 crypto/fips140/fips140-crypto-module-marker.h create mode 100644 crypto/fips140/fips140-fn-redirect.c create mode 100644 crypto/fips140/fips140-loader.c create mode 100644 crypto/fips140/fips140-module.c create mode 100644 crypto/fips140/fips140-module.h create mode 100644 crypto/fips140/fips140-var-redirect.c create mode 100644 crypto/fips140/fips140.lds create mode 100755 crypto/fips140/gen-fips140-fn-redirect.sh create mode 100644 include/crypto/fips140-fn-redirect.h create mode 100644 include/crypto/fips140-redirect.h
Signature · 1 line
2.47.3
PATCH v2 1/19 crypto: add Kconfig options for standalone crypto module
Add Kconfig option CRYPTO_FIPS140_EXTMOD to enable standalone crypto module support that can override built-in cryptographic implementations.
Currently supports X86_64 and ARM64 architectures and requires CRYPTO and MODULES to be enabled.
--- crypto/Kconfig | 1 + crypto/fips140/Kconfig | 15 +++++++++++++++ 2 files changed, 16 insertions(+) create mode 100644 crypto/fips140/Kconfig
crypto/Kconfig +1 -0
@@ -1387,6 +1387,7 @@ endif endif source "drivers/crypto/Kconfig" +source "crypto/fips140/Kconfig" source "crypto/asymmetric_keys/Kconfig" source "certs/Kconfig" source "crypto/krb5/Kconfig"
crypto/fips140/Kconfig new file +15 -0
@@ -0,0 +1,15 @@ +config CRYPTO_FIPS140_EXTMOD + bool "FIPS 140 compliant algorithms as a kernel module" + depends on CRYPTO && (X86_64 || ARM64) && MODULES + select CRYPTO_FIPS + help + This option enables building a kernel module that contains + copies of crypto algorithms that are built in a way that + complies with the FIPS 140 standard. + + The module registers the algorithms it contains with the + kernel crypto API, and the kernel crypto API's FIPS 140 mode + can be enabled to restrict crypto algorithm usage to only + those provided by this module. + + If unsure, say N.
Signature · 1 line
2.47.3
PATCH v2 2/19 crypto: add module entry for standalone crypto kernel module
In reply to Jay WangAdd the module entry for the standalone FIPS kernel crypto module. This creates the basic structure for fips140.ko that will be linked with built-in crypto implementations in later patches.
The implementation includes module initialization and exit functions and add into build system.
--- crypto/Makefile | 5 +++++ crypto/fips140/Makefile | 3 +++ crypto/fips140/fips140-module.c | 36 +++++++++++++++++++++++++++++++++ crypto/fips140/fips140-module.h | 14 +++++++++++++ 4 files changed, 58 insertions(+) create mode 100644 crypto/fips140/Makefile create mode 100644 crypto/fips140/fips140-module.c create mode 100644 crypto/fips140/fips140-module.h
crypto/Makefile +5 -0
@@ -207,3 +207,8 @@ obj-$(CONFIG_CRYPTO_KDF800108_CTR) += kdf_sp800108.o obj-$(CONFIG_CRYPTO_DF80090A) += df_sp80090a.o obj-$(CONFIG_CRYPTO_KRB5) += krb5/ + +# FIPS 140 kernel module +obj-$(CONFIG_CRYPTO_FIPS140_EXTMOD) += fips140/ + +
crypto/fips140/Makefile new file +3 -0
@@ -0,0 +1,3 @@ + + +
crypto/fips140/fips140-module.c new file +36 -0
@@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* FIPS 140 Kernel Cryptographic Module +* +* This file is the module entry point for fips140.ko, which is linked with previously built-in cryptos +* to generate the fips140.ko module. +* At load time, this module plugs the previously built-in implementations contained within itself back to the kernel. +* It also runs self-tests on these algorithms and verifies the integrity of its code and data. +* If either of these steps fails, the kernel will panic. +*/ + +#include "fips140-module.h" + +#define FIPS140_MODULE_NAME "FIPS 140 Kernel Cryptographic Module" +#define FIPS140_MODULE_VERSION "1.0.0" + +#define CRYPTO_INTERNAL "CRYPTO_INTERNAL" + +/* Initialize the FIPS 140 module */ +static int __init fips140_init (void ) +{ +return 0 ; +} + +static void __exit fips140_exit(void) +{ +pr_info ("Unloading " FIPS140_MODULE_NAME "\n" ); +} + +module_init (fips140_init); +module_exit (fips140_exit); + +MODULE_IMPORT_NS (CRYPTO_INTERNAL ); +MODULE_LICENSE ("GPL v2" ); +MODULE_DESCRIPTION (FIPS140_MODULE_NAME ); +MODULE_VERSION (FIPS140_MODULE_VERSION );
crypto/fips140/fips140-module.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* FIPS 140 Kernel Cryptographic Module - Header File +*/ + +#ifndef _CRYPTO_FIPS140_MODULE_H +#define _CRYPTO_FIPS140_MODULE_H + +#include <linux/module.h> +#include <linux/crypto.h> +#include <crypto/algapi.h> +#include <linux/init.h> + +#endif /* _CRYPTO_FIPS140_MODULE_H */
Signature · 1 line
2.47.3
PATCH v2 3/19 build: special compilation rule for building the standalone crypto module
In reply to Jay WangAdd special build rules to redirect builtin crypto algorithms into a standalone module fips140.ko instead of the main kernel, by introducing a new crypto-objs-y compilation rule that collects crypto objects to fips140.ko when CONFIG_CRYPTO_FIPS140_EXTMOD is enabled. Details as below:
Originally, builtin crypto (i.e., which specified by CONFIG_CRYPTO_{NAME}=y) are compiled into main kernel by default. On the contrary, this standalone crypto module feature requires compiling builtin crypto into modules while ensuring such cryptos are not included into main kernel anymore.
A naive way is to have a separate makefile containing all compilation rules for all cryptos for such standalone module, but such makefile could be too large to be scalable to arbitrary CONFIG_CRYPTO_{NAME} configuration, plus, this method cannot ensure the builtin crypto is completely removed from main kernel either.
To tackle this challenge, this patch automates object linking redirection by introducing special build logic for kernel crypto subsystem. Specifically:
First, to automatically collect crypto objects file while preserving the original compilation setting (such as flags, headers, path etc), it introduces a specific compilation rule crypto-objs-y += *.o that replaces the original rule obj-y += *.o in the crypto subsystem Makefile. As a result, when the standalone crypto module feature is turned on, for any crypto chosen to be builtin (e.g., crypto-objs-$(CONFIG_CRYPTO_SKCIPHER2) += *.o where CONFIG_CRYPTO_SKCIPHER2=y), it will be automatically collected and linked into a final object binary: fips140.o, with special compilation flag (-DFIPS_MODULE=1) to tell each individual obj files compiled in a specific way (will be used in later patches for generating pluggable interface on main kernel side and module side respectively from same source code). The implementation details are: it refers to the methodology of how obj-y collecting method of vmlinux.o, it places the crypto-objs-y rule in scripts/Makefile.build that will be run in each directory Makefile, and create crypto-module.a that comprises all captured crypto-objs-y += *.o under same folders. In its parent folders, such crypto-module.a will be included into parents' crypto-module.a recursively and eventually used to generate final fips140.o.
Second, to generate the final kernel module fips140.ko with the above fips140.o, A naïve approach would be to directly inject the fips140.ko module build into the existing modules generation pipeline (i.e., make modules) by providing our pre-generated fips140.o. However, we choose not to do this because it would create a circular make rule dependency (which is invalid in Makefiles and causes build failures), resulting in mutual dependencies between the modules and vmlinux targets (i.e., modules:vmlinux and vmlinux:modules at the same time). This happens for the following reasons:
1. Since we will later embed fips140.ko into the final kernel image (as described in the later patch), we must make vmlinux depend on fips140.ko. In other words: vmlinux: fips140.ko.
2. When the kernel is built with CONFIG_DEBUG_INFO_BTF_MODULES=y, it
requires: modules: vmlinux. This is because CONFIG_DEBUG_INFO_BTF_MODULES=y will take vmlinux as input to generate a BTF info for the module, and insert such info into the .ko module by default.
3. If we choose to inject fips140.ko into make modules, this would create a make rule dependency: fips140.ko: modules. Combined with items 1 and 2, this eventually creates an invalid circular dependency between vmlinux and modules.
Due to these reasons, the design choice is to use a separate make pipeline (defined as fips140-ready in the Makefile). This new pipeline reuses the same module generation scripts used by make modules but adds additional logic in scripts/Makefile.{modfinal|modinst|modpost} and scripts/mod/modpost.c to handle module symbol generation and verification correctly. In such pipeline, to eliminate the make rule dependency of fips140.ko:vmlinux which imposed by CONFIG_DEBUG_INFO_BTF_MODULES=y, we temporarily not generating BTF info for fips140.ko in this patch, where the BTF info for fips140.ko will be generated in later patch in a special manner.
Finally, with these special build rules, crypto/fips140/fips140.ko is generated.
--- Makefile | 34 +++++++++++- crypto/fips140/Makefile | 4 +- scripts/Makefile.build | 109 +++++++++++++++++++++++++++++++++++++- scripts/Makefile.modfinal | 22 ++++++++ scripts/Makefile.modinst | 13 ++++- scripts/Makefile.modpost | 25 +++++++++ scripts/mod/modpost.c | 24 +++++++-- 7 files changed, 223 insertions(+), 8 deletions(-)
Makefile +33 -1
@@ -1329,6 +1329,38 @@ PHONY += vmlinux # vmlinux: private export LDFLAGS_vmlinux := $(LDFLAGS_vmlinux) vmlinux: private _LDFLAGS_vmlinux := $(LDFLAGS_vmlinux) vmlinux: export LDFLAGS_vmlinux = $(_LDFLAGS_vmlinux) +ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +vmlinux: fips140-ready +# Ensure fips140.ko is built before embedding +fips140-ready: crypto/fips140/fips140.o crypto/fips140/.fips140.order crypto/fips140/fips140.mod vmlinux.o | modules_prepare + $(Q)$(MAKE) KBUILD_MODULES= -f $(srctree)/scripts/Makefile.modpost + $(Q)$(MAKE) KBUILD_MODULES=y crypto-module-gen=1 -f $(srctree)/scripts/Makefile.modpost +ifneq ($(KBUILD_MODPOST_NOFINAL),1) + $(Q)$(MAKE) KBUILD_MODULES=y crypto-module-gen=1 -f $(srctree)/scripts/Makefile.modfinal +endif + @: + +# Generate fips140.o from crypto-module.a files +crypto/fips140/fips140.o: crypto-module.a FORCE + $(call if_changed,ld_fips140) +crypto/fips140/.fips140.order: crypto/fips140/fips140.o + echo "crypto/fips140/fips140.o" > $@ +crypto/fips140/fips140.mod: crypto-module.a FORCE + $(call if_changed,fips140_mod) +crypto/fips140/.fips140.symvers: fips140-ready + @: +modpost: crypto/fips140/.fips140.symvers +quiet_cmd_ld_fips140 = LD [M] $@ + cmd_ld_fips140 = $(LD) -r $(KBUILD_LDFLAGS) $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE) --build-id=none --whole-archive $< --no-whole-archive -o $@ + +cmd_fips140_mod = ar -t $< > $@ + +# Add to targets so .cmd file is included +targets += crypto/fips140/fips140.o crypto/fips140/fips140.mod + +# Bridge rule: crypto-module.a depends on directory build (like built-in.a) +crypto-module.a: . ; +endif vmlinux: vmlinux.o $(KBUILD_LDS) modpost $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.vmlinux
@@ -2136,7 +2168,7 @@ prepare: outputmakefile # Error messages still appears in the original language PHONY += $(build-dir) $(build-dir): prepare - $(Q)$(MAKE) $(build)=$@ need-builtin=1 need-modorder=1 $(single-goals) + $(Q)$(MAKE) $(build)=$@ need-builtin=1 need-modorder=1 need-crypto=1 $(single-goals) clean-dirs := $(addprefix _clean_, $(clean-dirs)) PHONY += $(clean-dirs) clean
crypto/fips140/Makefile +3 -1
@@ -1,3 +1,5 @@ +crypto-objs-y += \ + fips140-module.o - +clean-files:= .fips140.order .fips140.symvers
scripts/Makefile.build +107 -2
206 lines not on this page · raw message
scripts/Makefile.modfinal +22 -0
@@ -11,7 +11,15 @@ include $(srctree)/scripts/Kbuild.include include $(srctree)/scripts/Makefile.lib # find all modules listed in modules.order +ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +ifeq ($(crypto-module-gen),1) +modules := $(call read-file, crypto/fips140/.fips140.order) +else modules := $(call read-file, modules.order) +endif +else +modules := $(call read-file, modules.order) +endif __modfinal: $(modules:%.o=%.ko) @:
@@ -55,12 +63,26 @@ if_changed_except = $(if $(call newer_prereqs_except,$(2))$(cmd-check), \ printf '%s\n' 'savedcmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:) # Re-generate module BTFs if either module's .ko or vmlinux changed +ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +ifeq ($(crypto-module-gen),1) +%.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds FORCE + +$(call if_changed,ld_ko_o) +else %.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE +$(call if_changed_except,ld_ko_o,$(objtree)/vmlinux) ifdef CONFIG_DEBUG_INFO_BTF_MODULES +$(if $(newer-prereqs),$(call cmd,btf_ko)) endif +$(call cmd,check_tracepoint) +endif +else +%.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE + +$(call if_changed_except,ld_ko_o,$(objtree)/vmlinux) +ifdef CONFIG_DEBUG_INFO_BTF_MODULES + +$(if $(newer-prereqs),$(call cmd,btf_ko)) +endif + +$(call cmd,check_tracepoint) +endif targets += $(modules:%.o=%.ko) $(modules:%.o=%.mod.o) .module-common.o
scripts/Makefile.modinst +12 -1
@@ -28,7 +28,12 @@ $(MODLIB)/modules.order: modules.order FORCE $(call cmd,install_modorder) quiet_cmd_install_modorder = INSTALL $@ - cmd_install_modorder = sed 's:^\(.*\)\.o$$:kernel/\1.ko:' $< > $@ + cmd_install_modorder = sed 's:^\(.*\)\.o$$:kernel/\1.ko:' $< > $@; \ + $(if $(CONFIG_CRYPTO_FIPS140_EXTMOD), \ + if [ -f crypto/fips140/.fips140.order ]; then \ + sed 's:^\(.*\)\.o$$:kernel/\1.ko:' crypto/fips140/.fips140.order >> $@; \ + fi \ + ) # Install modules.builtin(.modinfo,.ranges) even when CONFIG_MODULES is disabled. install-y += $(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo)
@@ -42,6 +47,12 @@ endif modules := $(call read-file, modules.order) +ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +ifneq ($(wildcard crypto/fips140/.fips140.order),) +modules += $(call read-file, crypto/fips140/.fips140.order) +endif +endif + ifeq ($(KBUILD_EXTMOD),) dst := $(MODLIB)/kernel else
scripts/Makefile.modpost +25 -0
@@ -51,6 +51,7 @@ modpost-args = \ $(if $(KBUILD_NSDEPS),-d modules.nsdeps) \ $(if $(CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS)$(KBUILD_NSDEPS),-N) \ $(if $(findstring 1, $(KBUILD_EXTRA_WARN)),-W) \ + $(if $(crypto-module-gen),-c) \ -o $@ modpost-deps := $(MODPOST)
@@ -63,10 +64,22 @@ endif # Read out modules.order to pass in modpost. # Otherwise, allmodconfig would fail with "Argument list too long". ifdef KBUILD_MODULES +ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +ifeq ($(crypto-module-gen),1) +# For crypto-module-gen, use .fips140.order instead of modules.order +modpost-args += -T crypto/fips140/.fips140.order +modpost-deps += crypto/fips140/.fips140.order +else +modpost-args += -T modules.order +modpost-deps += modules.order +endif +else modpost-args += -T modules.order modpost-deps += modules.order endif +endif + ifeq ($(KBUILD_EXTMOD),) # Generate the list of in-tree objects in vmlinux
@@ -110,6 +123,18 @@ modpost-deps += vmlinux.o output-symdump := $(if $(KBUILD_MODULES), Module.symvers, vmlinux.symvers) endif +# Include .fips140.symvers for regular module builds if it exists +ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +ifeq ($(KBUILD_MODULES),y) +ifneq ($(crypto-module-gen),1) +ifneq ($(wildcard crypto/fips140/.fips140.symvers),) +modpost-args += -i crypto/fips140/.fips140.symvers +modpost-deps += crypto/fips140/.fips140.symvers +endif +endif +endif +endif + else # set src + obj - they may be used in the modules's Makefile
scripts/mod/modpost.c +21 -3
@@ -43,6 +43,8 @@ static bool extended_modversions;static bool external_module;/* Only warn about unresolved symbols */ static bool warn_unresolved; +/* Crypto module generation mode */ +static bool crypto_module_gen;static int sec_mismatch_count;static bool sec_mismatch_warn_only = true;
@@ -1751,6 +1753,9 @@ static void check_exports(struct module *mod)const char *basename; exp =find_symbol (s->name);if (!exp) { +/* Skip undefined symbol errors for crypto module generation */ +if (crypto_module_gen) +continue ;if (!s->weak && nr_unresolved++ <MAX_UNRESOLVED_REPORTS ) modpost_log(!warn_unresolved, "\"%s\" [%s.ko] undefined!\n",
@@ -2210,10 +2215,14 @@ static void write_dump(const char *fname)struct buffer buf = { };struct module *mod;struct symbol *sym; +bool is_fips_symvers = crypto_module_gen && (strcmp (fname,"crypto/fips140/.fips140.symvers" ) ==0 );list_for_each_entry (mod, &modules, list) {if (mod->dump_file)continue ; +/* Skip vmlinux symbols when writing .fips140.symvers */ +if (is_fips_symvers && mod->is_vmlinux) +continue ;list_for_each_entry (sym, &mod->exported_symbols, list) {if (trim_unused_exports && !sym->used)continue ;
@@ -2285,7 +2294,7 @@ int main(int argc, char **argv)LIST_HEAD (dump_lists);struct dump_list *dl, *dl2; -while ((opt =getopt (argc, argv,"ei:MmnT:to:au:WwENd:xb" )) !=-1 ) { +while ((opt =getopt (argc, argv,"ei:MmnT:to:au:WwENd:xbc" )) !=-1 ) {switch (opt) {case 'e' : external_module = true;
@@ -2340,6 +2349,9 @@ int main(int argc, char **argv)case 'x' : extended_modversions = true;break ; +case 'c' : + crypto_module_gen = true; +break ;default :exit (1 ); }
@@ -2385,8 +2397,14 @@ int main(int argc, char **argv)if (missing_namespace_deps)write_namespace_deps_files (missing_namespace_deps); -if (dump_write) -write_dump (dump_write); +if (dump_write) { +if (crypto_module_gen) { +/* generate .fips140.symvers for FIPS crypto modules */ +write_dump ("crypto/fips140/.fips140.symvers" ); + }else { +write_dump (dump_write); + } + }if (sec_mismatch_count && !sec_mismatch_warn_only)error ("Section mismatches detected.\n" "Set CONFIG_SECTION_MISMATCH_WARN_ONLY=y to allow them.\n" );
Signature · 1 line
2.47.3
PATCH v2 4/19 build: Add ELF marker for crypto-objs-m modules
In reply to Jay WangPreviously, crypto-objs-$(CONFIG_*) behavior depends on the config value. When CONFIG_*=y, crypto is built into fips140.ko. When CONFIG_*=m, crypto is already built as a separate module (e.g., aes.ko), so previous patches do not affect such modules.
This patch adds an ELF marker to identify modules built with CONFIG_*=m so they can be distinguished as part of the CONFIG_CRYPTO_FIPS140_EXTMOD framework. This gives module loaders a way to tell the module is included in crypto-objs-m.
--- crypto/fips140/fips140-crypto-module-marker.h | 8 ++++++++ scripts/Makefile.build | 15 +++++++++++++++ 2 files changed, 23 insertions(+) create mode 100644 crypto/fips140/fips140-crypto-module-marker.h
crypto/fips140/fips140-crypto-module-marker.h new file +8 -0
@@ -0,0 +1,8 @@ +#ifndef _FIPS140_CRYPTO_MODULE_MARKER_H +#define _FIPS140_CRYPTO_MODULE_MARKER_H + +/* Crypto module marker - automatically included for crypto-objs-m modules */ +static const char __fips140_crypto_marker[] + __attribute__((section(".fips140_crypto_marker"), used)) = "FIPS140_CRYPTO_OBJS_M"; + +#endif /* _FIPS140_CRYPTO_MODULE_MARKER_H */
scripts/Makefile.build +15 -0
@@ -69,6 +69,7 @@ obj-m += $(crypto-objs-m) ifndef CONFIG_CRYPTO_FIPS140_EXTMOD obj-y += $(crypto-objs-y) crypto-objs-y := +crypto-objs-m := $(filter-out $(crypto-objs-y),$(crypto-objs-m)) endif # When an object is listed to be built compiled-in and modular,
@@ -131,6 +132,7 @@ multi-obj-m := $(call multi-search, $(obj-m), .o, -objs -y -m) multi-obj-ym := $(multi-obj-y) $(multi-obj-m) ifdef CONFIG_CRYPTO_FIPS140_EXTMOD multi-crypto-objs-y := $(call multi-search, $(crypto-objs-y), .o, -objs -y) +multi-crypto-objs-m := $(call multi-search, $(crypto-objs-m), .o, -objs -y -m) endif # Replace multi-part objects by their individual parts,
@@ -139,6 +141,7 @@ real-obj-y := $(call real-search, $(obj-y), .o, -objs -y) real-obj-m := $(call real-search, $(obj-m), .o, -objs -y -m) ifdef CONFIG_CRYPTO_FIPS140_EXTMOD real-crypto-objs-y := $(strip $(call real-search, $(crypto-objs-y), .o, -objs -y)) +real-crypto-objs-m := $(strip $(call real-search, $(crypto-objs-m), .o, -objs -y -m)) endif always-y += $(always-m)
@@ -166,11 +169,13 @@ real-obj-y := $(addprefix $(obj)/, $(real-obj-y)) real-obj-m := $(addprefix $(obj)/, $(real-obj-m)) ifdef CONFIG_CRYPTO_FIPS140_EXTMOD real-crypto-objs-y := $(addprefix $(obj)/, $(real-crypto-objs-y)) +real-crypto-objs-m := $(addprefix $(obj)/, $(real-crypto-objs-m)) endif multi-obj-m := $(addprefix $(obj)/, $(multi-obj-m)) subdir-ym := $(addprefix $(obj)/, $(subdir-ym)) ifdef CONFIG_CRYPTO_FIPS140_EXTMOD multi-crypto-objs-y := $(addprefix $(obj)/, $(multi-crypto-objs-y)) +multi-crypto-objs-m := $(addprefix $(obj)/, $(multi-crypto-objs-m)) endif endif
@@ -578,6 +583,16 @@ $(multi-crypto-objs-y): %.o: %.mod FORCE $(call multi_depend, $(multi-crypto-objs-y), .o, -objs -y -m) endif endif + +# Individual object compilation with version-specific flags +$(real-crypto-objs-m): private KBUILD_CFLAGS += -DFIPS140_CRYPTO_OBJS_M=1 -include $(srctree)/crypto/fips140/fips140-crypto-module-marker.h + +# Also set flags for individual objects that make up composite crypto objects +$(foreach obj,$(multi-crypto-objs-m),$($(obj:.o=-y))): private KBUILD_CFLAGS += -DFIPS140_CRYPTO_OBJS_M=1 +$(foreach obj,$(multi-crypto-objs-m),$($(obj:.o=-objs))): private KBUILD_CFLAGS += -DFIPS140_CRYPTO_OBJS_M=1 + +# Multi-part crypto objects +$(multi-crypto-objs-m): private KBUILD_CFLAGS += -DFIPS140_CRYPTO_OBJS_M=1 -include $(srctree)/crypto/fips140/fips140-crypto-module-marker.h endif # This is a list of build artifacts from the current Makefile and its # sub-directories. The timestamp should be updated when any of the member files.
Signature · 1 line
2.47.3
PATCH v2 5/19 module: allow kernel module loading directly from memory
In reply to Jay WangFrom: Vegard Nossum <[email protected]>
To enable loading the crypto module earlier before file system is ready, add a new helper function, load_crypto_module_mem(), which can load a kernel module from a byte array in memory. When loading in this way, we don't do signature verification as crypto is not ready yet before loaded. To tell that a module is loaded in this way, a new module loader flag, MODULE_INIT_CRYPTO_FROM_MEM, is added.
--- include/linux/module.h | 2 + include/uapi/linux/module.h | 5 ++ kernel/module/main.c | 100 +++++++++++++++++++++++++----------- kernel/params.c | 3 +- 4 files changed, 79 insertions(+), 31 deletions(-)
include/linux/module.h +2 -0
@@ -588,6 +588,8 @@ struct module {#ifdef CONFIG_MODULES +extern int load_crypto_module_mem (const char *mem,size_t size); +/* Get/put a kernel symbol (calls must be symmetric) */ void *__symbol_get (const char *symbol);void *__symbol_get_gpl (const char *symbol);
include/uapi/linux/module.h +5 -0
@@ -7,4 +7,9 @@#define MODULE_INIT_IGNORE_VERMAGIC 2#define MODULE_INIT_COMPRESSED_FILE 4 +#ifdef __KERNEL__ +/* Internal flags */ +#define MODULE_INIT_CRYPTO_FROM_MEM (1 << 8) +#endif +#endif /* _UAPI_LINUX_MODULE_H */
kernel/module/main.c +70 -30
161 lines not on this page · raw message
kernel/params.c +2 -1
@@ -957,7 +957,8 @@ static int __init param_sysfs_init(void)return 0 ; } -subsys_initcall (param_sysfs_init); +/* Use arch_initcall instead of subsys_initcall for early module loading */ +arch_initcall (param_sysfs_init);/* * param_sysfs_builtin_init - add sysfs version and parameter
Signature · 1 line
2.47.3
PATCH v2 6/19 crypto: add pluggable interface for module symbols referenced by the main kernel
In reply to Jay WangThe main kernel and modules interact through exported symbols (EXPORT_SYMBOL). When built-in cryptographic algorithms are moved into a standalone kernel module (fips140.ko), the crypto functions and variables must be exported from the module so the main kernel can use them. However, the existing module symbol resolution is one-way:
it supports symbols defined in the main kernel and referenced by modules, but not the reverse — symbols defined in a module but referenced by the main kernel. Since the linker requires all symbol addresses to be resolved at link time, moving crypto symbols out of vmlinux would break compilation.
To address this, introduce a pluggable interface that places address placeholders at all crypto usage points in the main kernel. These placeholders are initially set to NULL during compilation to satisfy the linker. At runtime, once fips140.ko is loaded, the placeholders are updated to the correct addresses before their first use.
Two types of address placeholders are used. For function symbols (the majority), a trampoline (naked function) tail-jumps through a function pointer marked __ro_after_init to prevent modification after kernel init. This is implemented as DEFINE_CRYPTO_FN_REDIRECT(). For variable symbols (a smaller number), a pointer of the corresponding data type serves as the placeholder. This is implemented as DECLARE_CRYPTO_VAR() for declarations and DEFINE_CRYPTO_VAR_STUB() for definitions. These wrappers are compiled differently for the main kernel and for the crypto module source code — acting as the "outlet" and the "plug" respectively — using the -DFIPS_MODULE compilation flag.
To apply these placeholders to a crypto symbol, the main kernel must be directed to use the placeholder instead of the original address. Since all crypto users include the corresponding header files, the headers are a natural place to perform this redirection.
For exported variable symbols (a small number, ~10), the declaration in the header file is replaced with the DECLARE_CRYPTO_VAR() wrapper, and the placeholder definition DEFINE_CRYPTO_VAR_STUB() is added to a dedicated file fips140-var-redirect.c (applied in later patches). The wrapper takes the Kconfig symbol as a parameter so that when a crypto algorithm is already built as a module, the original declaration remains unchanged.
For exported function symbols (the majority, ~hundreds), the key design goal is that no kernel source tree modification is needed at all. Instead of manually modifying header files, the redirection is fully automated during the build process. The exported symbol list is extracted from fips140.o into .fips140.exported, then gen-fips140-fn-redirect.sh auto-generates DEFINE_CRYPTO_FN_REDIRECT() calls for each symbol. link-vmlinux.sh adds --wrap=<sym> linker flags so all references in vmlinux are redirected to __wrap_<sym> trampolines. EXPORT_SYMBOL in the FIPS module emits __crypto_fn_keys entries that map each __fips140_fn_ptr_<sym> to the real function address. As a result, no header files or call sites need to be touched to redirect any crypto function.
At module load time, do_crypto_var() and do_crypto_fn() in kernel/module/main.c walk the __crypto_var_keys and __crypto_fn_keys sections respectively, writing the real addresses into the placeholders.
The pluggable interface idea originates from Vegard Nossum <[email protected]>. This implementation additionally provides automated conversion of crypto functions to pluggable redirects without massive kernel source tree changes, avoids duplicate crypto code in the main kernel, supports variable symbol redirection, does not interfere with cryptos already configured as modules, and adapts to arbitrary .config choices.
--- Makefile | 31 +++++++- crypto/fips140/Makefile | 5 +- crypto/fips140/fips140-fn-redirect.c | 10 +++ crypto/fips140/gen-fips140-fn-redirect.sh | 28 +++++++ include/asm-generic/vmlinux.lds.h | 2 + include/crypto/fips140-fn-redirect.h | 63 ++++++++++++++++ include/crypto/fips140-redirect.h | 92 +++++++++++++++++++++++ include/linux/export.h | 22 ++++++ kernel/module/main.c | 45 ++++++++++- scripts/link-vmlinux.sh | 12 +++ 10 files changed, 305 insertions(+), 5 deletions(-) create mode 100644 crypto/fips140/fips140-fn-redirect.c create mode 100755 crypto/fips140/gen-fips140-fn-redirect.sh create mode 100644 include/crypto/fips140-fn-redirect.h create mode 100644 include/crypto/fips140-redirect.h
Makefile +29 -2
@@ -1302,11 +1302,38 @@ endif quiet_cmd_ar_vmlinux.a = AR $@ cmd_ar_vmlinux.a = \ rm -f $@; \ - $(AR) cDPrST $@ $(KBUILD_VMLINUX_OBJS); \ + $(AR) cDPrST $@ $(KBUILD_VMLINUX_OBJS) $(fips140-fn-redirect-obj); \ $(AR) mPiT $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt) +ifdef CONFIG_CRYPTO_FIPS140_EXTMOD + +# Generate exported symbol list from fips140.o (no vmlinux.o dependency) +quiet_cmd_gen_fips140_exported = + cmd_gen_fips140_exported = $(NM) $< 2>/dev/null | \ + sed -n 's/.*__export_symbol_//p' | sort | \ + awk '{print "0x00000000\t" $$1 "\tcrypto/fips140/fips140\tEXPORT_SYMBOL_GPL\t"}' > $@ + +crypto/fips140/.fips140.exported: crypto/fips140/fips140.o FORCE + $(call if_changed,gen_fips140_exported) + +# Generate fn-redirect header from exported symbol list +quiet_cmd_gen_fips140_fn_redirect = + cmd_gen_fips140_fn_redirect = $(CONFIG_SHELL) $(srctree)/crypto/fips140/gen-fips140-fn-redirect.sh \ + crypto/fips140/.fips140.exported $@ + +crypto/fips140/.fips140-fn-redirect.h: crypto/fips140/.fips140.exported + $(call cmd,gen_fips140_fn_redirect) + +crypto/fips140/fips140-fn-redirect.o: crypto/fips140/.fips140-fn-redirect.h $(srctree)/crypto/fips140/fips140-fn-redirect.c + $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.build obj=crypto/fips140 $@ + +fips140-fn-redirect-obj := crypto/fips140/fips140-fn-redirect.o + +targets += crypto/fips140/.fips140.exported crypto/fips140/.fips140-fn-redirect.h +endif + targets += vmlinux.a -vmlinux.a: $(KBUILD_VMLINUX_OBJS) scripts/head-object-list.txt FORCE +vmlinux.a: $(KBUILD_VMLINUX_OBJS) $(fips140-fn-redirect-obj) scripts/head-object-list.txt FORCE $(call if_changed,ar_vmlinux.a) PHONY += vmlinux_o
crypto/fips140/Makefile +4 -1
@@ -2,4 +2,7 @@ crypto-objs-y += \ fips140-module.o -clean-files:= .fips140.order .fips140.symvers +CFLAGS_fips140-fn-redirect.o += -I$(obj) +CFLAGS_fips140-module.o += -DFIPS140_CORE + +clean-files:= .fips140.order .fips140.symvers .fips140-fn-redirect.h .fips140.exported
crypto/fips140/fips140-fn-redirect.c new file +10 -0
@@ -0,0 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/kernel.h> +#include <crypto/fips140-fn-redirect.h> +#include ".fips140-fn-redirect.h" + +void __fips140_fn_not_redirected (void ) +{ +panic ("FIPS140: redirected function called before fips140.ko loaded!\n" ); +} +
crypto/fips140/gen-fips140-fn-redirect.sh new file +28 -0
@@ -0,0 +1,28 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Generate DEFINE_CRYPTO_FN_REDIRECT() calls for all exported symbols +# +# Usage: gen-fips140-fn-redirect.sh <.fips140.exported> <output.h> + +EXPORTED="$1" +OUTPUT="$2" + +if [ ! -f "$EXPORTED" ]; then + echo "/* exported symbol list not found, empty redirect list */" > "$OUTPUT" + exit 0 +fi + +TMPFILE="${OUTPUT}.tmp" + +{ + echo "/* SPDX-License-Identifier: GPL-2.0 */" + echo "/* Auto-generated by gen-fips140-fn-redirect.sh — do not edit */" + echo "" + awk '{print "DEFINE_CRYPTO_FN_REDIRECT(" $2 ")"}' "$EXPORTED" +} > "$TMPFILE" + +if [ -f "$OUTPUT" ] && cmp -s "$TMPFILE" "$OUTPUT"; then + rm -f "$TMPFILE" +else + mv "$TMPFILE" "$OUTPUT" +fi
include/asm-generic/vmlinux.lds.h +2 -0
@@ -717,6 +717,8 @@ KERNEL_CTORS() \MCOUNT_REC () \ *(.init.rodata .init.rodata.*) \ + BOUNDED_SECTION(__crypto_var_keys) \ + BOUNDED_SECTION(__crypto_fn_keys) \ FTRACE_EVENTS() \ TRACE_SYSCALLS() \ KPROBE_BLACKLIST() \
include/crypto/fips140-fn-redirect.h new file +63 -0
63 lines not on this page · raw message
include/crypto/fips140-redirect.h new file +92 -0
92 lines not on this page · raw message
include/linux/export.h +22 -0
@@ -72,11 +72,33 @@#define __GENDWARFKSYMS_EXPORT (sym)#endif +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && defined(FIPS_MODULE ) && !defined(FIPS140_CORE ) +struct _crypto_fn_key { +void **ptr; +void *func; +}; +#define __CRYPTO_FN_KEY (sym) \ + extern void *__fips140_fn_ptr_##sym; \ + static struct _crypto_fn_key __##sym##_fn_key \ + __used \ + __section("__crypto_fn_keys") \ + __aligned(__alignof__(struct _crypto_fn_key)) = { \ + .ptr = (void **)&__fips140_fn_ptr_##sym, \ + .func = (void *)&sym, \ + }; +#define __EXPORT_SYMBOL (sym, license, ns) \ + extern typeof(sym) sym; \ + __ADDRESSABLE(sym) \ + __GENDWARFKSYMS_EXPORT(sym) \ + asm(__stringify(___EXPORT_SYMBOL(sym, license, ns))); \ + __CRYPTO_FN_KEY(sym) +#else #define __EXPORT_SYMBOL (sym, license, ns) \ extern typeof(sym) sym; \ __ADDRESSABLE(sym) \ __GENDWARFKSYMS_EXPORT(sym) \ asm(__stringify(___EXPORT_SYMBOL(sym, license, ns))) +#endif #endif
kernel/module/main.c +43 -2
75 lines not on this page · raw message
scripts/link-vmlinux.sh +12 -0
@@ -91,6 +91,18 @@ vmlinux_link() ldflags="${ldflags} ${wl}--script=${objtree}/${KBUILD_LDS}" + if is_enabled CONFIG_CRYPTO_FIPS140_EXTMOD; then + local fips_ko=crypto/fips140/fips140.ko + local fips_exported=crypto/fips140/.fips140.exported + if [ -f "${fips_ko}" ] && [ -f "${fips_exported}" ]; then + for sym in $(awk '{print $2}' "${fips_exported}" | while read s; do + ${NM} "${fips_ko}" 2>/dev/null | awk -v s="$s" '$3 == s && $2 == "T" {print s; exit}' + done); do + ldflags="${ldflags} ${wl}--wrap=${sym}" + done + fi + fi + # The kallsyms linking does not need debug symbols included. if [ -n "${strip_debug}" ] ; then ldflags="${ldflags} ${wl}--strip-debug"
Signature · 1 line
2.47.3
PATCH v2 7/19 crypto: dedicated ELF sections for collected crypto initcalls
In reply to Jay WangCryptographic components must be properly initialized before use. This initialization is typically achieved through dedicated init functions registered via wrappers such as module_init() or late_initcall(). Traditionally, these init functions are executed automatically as part of the kernel boot sequence. However, now that the crypto code is moved into a standalone module (fips140.ko), there needs to be a way to collect and later execute them from within the module.
To collect these init functions, the init wrappers (module_init(), subsys_initcall(), late_initcall()) are modified so that when compiled for the FIPS module (under -DFIPS_MODULE), they automatically place the wrapped crypto init function pointer into a dedicated ELF section instead of the normal initcall mechanism. A custom linker script crypto/fips140/fips140.lds is introduced to organize these sections. Since the init functions must be called in proper ordering in a later patch (e.g., subsys_initcall before module_init, and module_init before late_initcall), the linker script allocates separate leveled sections (.fips_initcall0, .fips_initcall1, .fips_initcall2) with corresponding boundary symbols (e.g., __fips140_initcall0_start/end) to preserve the correct execution order.
--- Makefile | 2 +- crypto/fips140/fips140.lds | 38 ++++++++++++++++++++++++++++++++++++++ include/linux/module.h | 23 +++++++++++++++++++++++ 3 files changed, 62 insertions(+), 1 deletion(-) create mode 100644 crypto/fips140/fips140.lds
Makefile +1 -1
@@ -1378,7 +1378,7 @@ crypto/fips140/.fips140.symvers: fips140-ready @: modpost: crypto/fips140/.fips140.symvers quiet_cmd_ld_fips140 = LD [M] $@ - cmd_ld_fips140 = $(LD) -r $(KBUILD_LDFLAGS) $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE) --build-id=none --whole-archive $< --no-whole-archive -o $@ + cmd_ld_fips140 = $(LD) -r $(KBUILD_LDFLAGS) $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE) --build-id=none -T $(srctree)/crypto/fips140/fips140.lds --whole-archive $< --no-whole-archive -o $@ cmd_fips140_mod = ar -t $< > $@
crypto/fips140/fips140.lds new file +38 -0
@@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +/* + * FIPS 140 module initcall section layout. + * + * The overridden subsys_initcall/module_init/late_initcall macros + * (include/linux/module.h) place function pointers into these + * sections when compiled with FIPS_MODULE defined. + * + * Section mapping: + * .fips_initcall0 <- subsys_initcall() + * Syncs with kernel subsys_initcall (initcall level 4) + * .fips_initcall1 <- module_init() + * Syncs with kernel device_initcall (initcall level 6) + * .fips_initcall2 <- late_initcall() + * Syncs with kernel late_initcall (initcall level 7) + * + * The fips140 loader thread (fips140-loader.c) starts at + * arch_initcall_sync (level 3) and run_initcalls() in + * fips140-module.c executes each level in order, synchronizing + * with the kernel's initcall progression via wait queues. + */ + +SECTIONS { + .init.data : { + __fips140_initcalls_start = .; + __fips140_initcall0_start = .; + *(.fips_initcall0) + __fips140_initcall0_end = .; + __fips140_initcall1_start = .; + *(.fips_initcall1) + __fips140_initcall1_end = .; + __fips140_initcall2_start = .; + *(.fips_initcall2) + __fips140_initcall2_end = .; + __fips140_initcalls_end = .; + } +}
include/linux/module.h +23 -0
@@ -115,18 +115,40 @@ extern void cleanup_module(void);#define postcore_initcall (fn) module_init(fn)#define postcore_initcall_sync (fn) module_init(fn)#define arch_initcall (fn) module_init(fn) +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && defined(FIPS_MODULE ) && !defined(FIPS140_CORE ) +#define subsys_initcall (fn) \ + static initcall_t __used __section(".fips_initcall0") \ + __fips_##fn = fn; +#else #define subsys_initcall (fn) module_init(fn) +#endif #define subsys_initcall_sync (fn) module_init(fn)#define fs_initcall (fn) module_init(fn)#define fs_initcall_sync (fn) module_init(fn)#define rootfs_initcall (fn) module_init(fn)#define device_initcall (fn) module_init(fn)#define device_initcall_sync (fn) module_init(fn) +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && defined(FIPS_MODULE ) && !defined(FIPS140_CORE ) +#define late_initcall (fn) \ + static initcall_t __used __section(".fips_initcall2") \ + __fips_##fn = fn; +#else #define late_initcall (fn) module_init(fn) +#endif #define late_initcall_sync (fn) module_init(fn)#define console_initcall (fn) module_init(fn) +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && defined(FIPS_MODULE ) && !defined(FIPS140_CORE ) +/* FIPS module: place init/exit in special sections for fips140 loader */ +#define module_init (initfn) \ + static initcall_t __used __section(".fips_initcall1") \ + __fips_##initfn = initfn; + +#define module_exit (exitfn) \ + static unsigned long __used __section(".fips_exitcall") \ + __fips_##exitfn = (unsigned long)&exitfn; +#else /* Each module must use one module_init(). */ #define module_init (initfn) \ static inline initcall_t __maybe_unused __inittest(void) \
@@ -142,6 +164,7 @@ extern void cleanup_module(void); void cleanup_module(void) __copy(exitfn) \ __attribute__((alias(#exitfn))); \ ___ADDRESSABLE(cleanup_module, __exitdata); +#endif /* CONFIG_CRYPTO_FIPS140_EXTMOD && FIPS_MODULE && !FIPS140_CORE */ #endif
Signature · 1 line
2.47.3
PATCH v2 8/19 crypto: fips140: add crypto module loader
In reply to Jay WangAdd a crypto module loader mechanism that loads a precompiled crypto kernel module that is embedded in vmlinux image directly from memory (whose address stored in _binary_fips140_ko_start/end) during early boot.
This is built based on Vegard Nossum <[email protected]> and Saeed Mirzamohammadi <[email protected]>, the fips_loader_init is picked up. But different from them, such loader is not executed as arch_initcall_sync(), but rather as a thread along main kernel init to ensure proper initialization sequencing (Details are in later patch).
--- crypto/fips140/fips140-loader.c | 55 +++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 crypto/fips140/fips140-loader.c
crypto/fips140/fips140-loader.c new file +55 -0
@@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* FIPS 140 Early Loader +*/ + +#include <linux/init.h> +#include <linux/module.h> +#include <linux/printk.h> +#include <linux/vmalloc.h> +#include <linux/string.h> +#include <linux/elf.h> +#include <linux/kthread.h> +#include <linux/wait.h> + +extern const u8 _binary_fips140_ko_start[]; +extern const u8 _binary_fips140_ko_end[]; +const u8 *_binary_crypto_ko_start; +EXPORT_SYMBOL_GPL (_binary_crypto_ko_start); +const u8 *_binary_crypto_ko_end; +EXPORT_SYMBOL_GPL (_binary_crypto_ko_end); + +/* Function to load crypto module from memory */ +extern int load_crypto_module_mem (const char *mem,size_t size); + +static void load_prepare (void ) +{ + _binary_crypto_ko_start = _binary_fips140_ko_start; + _binary_crypto_ko_end = _binary_fips140_ko_end; +} + +static int fips_loader_init (void ) +{ +load_prepare (); + +const void *ko_mem = _binary_crypto_ko_start; +size_t ko_size = _binary_crypto_ko_end - _binary_crypto_ko_start; +void *vmalloc_mem; +int ret; + +// Copy to vmalloc'd memory since load_module expects to free it + vmalloc_mem =vmalloc (ko_size); +if (!vmalloc_mem) { +pr_err ("FIPS140 loader: failed to allocate memory\n" ); +return -ENOMEM ; + } + +memcpy (vmalloc_mem, ko_mem, ko_size); + + ret =load_crypto_module_mem (vmalloc_mem, ko_size);// Skip signature check +if (ret) +panic ("FIPS140 loader: module loading error\n" ); + +vfree (vmalloc_mem);// Free after successful module loading +return ret; +}
Signature · 1 line
2.47.3
PATCH v2 9/19 build: embed the standalone crypto module into vmlinux
In reply to Jay WangAs mentioned in earlier patch, in order to load standalone crypto module in early boot before filesystem is ready, the module needs to be embedded into vmlinux image. This patch intends to make such embedded process a seamless process that will automatically trigger as building vmlinux (i.e., during make vmlinux). So it adds make dependency rule such that vmlinux will depend on the fips140.ko and its signature .fips140.hmac generation rule. It also modifies vmlinux link rule to finally link them with vmlinux.o.
The high level idea of embedding fips140.ko into vmlinux stems from Vegard Nossum <[email protected]>.
--- Makefile | 32 +++++++++++++++++++++++++++++--- arch/arm64/kernel/vmlinux.lds.S | 16 ++++++++++++++++ arch/x86/kernel/vmlinux.lds.S | 16 ++++++++++++++++ crypto/fips140/Kconfig | 29 +++++++++++++++++++++++++++++ crypto/fips140/Makefile | 4 +++- crypto/fips140/fips140-loader.c | 9 +++++++++ scripts/Makefile.modfinal | 18 +++++++++++++++++- scripts/Makefile.vmlinux | 6 +++++- scripts/link-vmlinux.sh | 5 +++++ 9 files changed, 129 insertions(+), 6 deletions(-)
Makefile +29 -3
@@ -1306,12 +1306,21 @@ quiet_cmd_ar_vmlinux.a = AR $@ $(AR) mPiT $$($(AR) t $@ | sed -n 1p) $@ $$($(AR) t $@ | grep -F -f $(srctree)/scripts/head-object-list.txt) ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +fips140_build = . +ifeq ($(CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE),y) +fips140_build = fips140_build +endif # Generate exported symbol list from fips140.o (no vmlinux.o dependency) quiet_cmd_gen_fips140_exported = cmd_gen_fips140_exported = $(NM) $< 2>/dev/null | \ sed -n 's/.*__export_symbol_//p' | sort | \ - awk '{print "0x00000000\t" $$1 "\tcrypto/fips140/fips140\tEXPORT_SYMBOL_GPL\t"}' > $@ + awk '{print "0x00000000\t" $$1 "\tcrypto/fips140/fips140\tEXPORT_SYMBOL_GPL\t"}' > $@ \ + $(fips140_cp_exported) + +ifeq ($(CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE),y) +fips140_cp_exported = ; cp "$(fips140_build)/crypto/fips140/.fips140.exported" crypto/fips140/.fips140.exported +endif crypto/fips140/.fips140.exported: crypto/fips140/fips140.o FORCE $(call if_changed,gen_fips140_exported)
@@ -1357,7 +1366,22 @@ PHONY += vmlinux vmlinux: private _LDFLAGS_vmlinux := $(LDFLAGS_vmlinux) vmlinux: export LDFLAGS_vmlinux = $(_LDFLAGS_vmlinux) ifdef CONFIG_CRYPTO_FIPS140_EXTMOD -vmlinux: fips140-ready +vmlinux: crypto/fips140/fips140-embedded.o crypto/fips140/fips140-digest.o +crypto/fips140/fips140-embedded.o: fips140-ready + @echo " LD $@" + @$(LD) -r -b binary -o $@ $(fips140_build)/crypto/fips140/fips140.ko + @$(OBJCOPY) --rename-section .data=.fips140_module_data $@ + +crypto/fips140/.fips140.hmac: crypto/fips140/fips140-embedded.o + @echo " HMAC $@" + @hmac_key=$$(awk -F'"' '/^CONFIG_CRYPTO_FIPS140_HMAC_KEY=/{print $$2}' .config); \ + openssl dgst -sha256 -hmac "$$hmac_key" -binary -out $@ $(fips140_build)/crypto/fips140/fips140.ko + +crypto/fips140/fips140-digest.o: crypto/fips140/.fips140.hmac + @echo " LD $@" + @$(LD) -r -b binary -o $@ crypto/fips140/.fips140.hmac + @$(OBJCOPY) --rename-section .data=.fips140_digest $@ + # Ensure fips140.ko is built before embedding fips140-ready: crypto/fips140/fips140.o crypto/fips140/.fips140.order crypto/fips140/fips140.mod vmlinux.o | modules_prepare $(Q)$(MAKE) KBUILD_MODULES= -f $(srctree)/scripts/Makefile.modpost
@@ -1365,7 +1389,9 @@ fips140-ready: crypto/fips140/fips140.o crypto/fips140/.fips140.order crypto/fip ifneq ($(KBUILD_MODPOST_NOFINAL),1) $(Q)$(MAKE) KBUILD_MODULES=y crypto-module-gen=1 -f $(srctree)/scripts/Makefile.modfinal endif - @: +ifeq ($(CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE),y) + cp "$(fips140_build)/crypto/fips140/fips140.ko" crypto/fips140/fips140.ko; +endif # Generate fips140.o from crypto-module.a files crypto/fips140/fips140.o: crypto-module.a FORCE
arch/arm64/kernel/vmlinux.lds.S +16 -0
@@ -201,6 +201,22 @@ SECTIONS /* everything from this point to __init_begin will be marked RO NX */ RO_DATA(PAGE_SIZE) +#ifdef CONFIG_CRYPTO_FIPS140_EXTMOD + /* FIPS 140 embedded module data */ + .fips140_embedded : { + . = ALIGN(8); + _binary_fips140_ko_start = .; + KEEP(*(.fips140_module_data)) + _binary_fips140_ko_end = .; + } + .fips140_digest : { + . = ALIGN(8); + _binary_fips140_hmac_start = .; + KEEP(*(.fips140_digest)) + _binary_fips140_hmac_end = .; + } +#endif + HYPERVISOR_RODATA_SECTIONS .got : { *(.got) }
arch/x86/kernel/vmlinux.lds.S +16 -0
@@ -171,6 +171,22 @@ SECTIONS RO_DATA(PAGE_SIZE) X86_ALIGN_RODATA_END +#ifdef CONFIG_CRYPTO_FIPS140_EXTMOD + /* FIPS 140 embedded module data */ + .fips140_embedded : AT(ADDR(.fips140_embedded) - LOAD_OFFSET) { + . = ALIGN(8); + _binary_fips140_ko_start = .; + KEEP(*(.fips140_module_data)) + _binary_fips140_ko_end = .; + } + .fips140_digest : AT(ADDR(.fips140_digest) - LOAD_OFFSET) { + . = ALIGN(8); + _binary_fips140_hmac_start = .; + KEEP(*(.fips140_digest)) + _binary_fips140_hmac_end = .; + } +#endif + /* Data */ .data : AT(ADDR(.data) - LOAD_OFFSET) { /* Start of data section */
crypto/fips140/Kconfig +29 -0
@@ -12,4 +12,33 @@ config CRYPTO_FIPS140_EXTMOD can be enabled to restrict crypto algorithm usage to only those provided by this module. + If unsure, say N. +config CRYPTO_FIPS140_HMAC_KEY + string "FIPS 140-3 external module HMAC key" + depends on CRYPTO_FIPS140_EXTMOD + default "The quick brown fox jumps over the lazy dog while the sphinx of black quartz judges my vow" + help + This is the HMAC key used to build and verify the integrity of + the FIPS module. + + Must be at least 80 characters. +config CRYPTO_FIPS140_EXTMOD_SOURCE + bool "Use external FIPS module source" + depends on CRYPTO_FIPS140_EXTMOD + default n + help + Use pre-built FIPS modules from an external build directory instead + of freshly built modules from the current kernel build. + + If N, the kernel uses freshly generated crypto modules from the + current build directory: + - crypto/fips140/fips140.ko + - crypto/aes.ko + - crypto/sha256.ko + + If Y, pre-built modules from fips140_build/ are used: + - fips140_build/crypto/fips140/fips140.ko + - fips140_build/crypto/aes.ko + - fips140_build/crypto/sha256.ko + If unsure, say N.
crypto/fips140/Makefile +3 -1
@@ -2,7 +2,9 @@ crypto-objs-y += \ fips140-module.o +obj-y += fips140-loader.o + CFLAGS_fips140-fn-redirect.o += -I$(obj) CFLAGS_fips140-module.o += -DFIPS140_CORE -clean-files:= .fips140.order .fips140.symvers .fips140-fn-redirect.h .fips140.exported +clean-files:= .fips140.order .fips140.symvers .fips140-fn-redirect.h .fips140.exported .fips140.hmac
crypto/fips140/fips140-loader.c +9 -0
@@ -14,10 +14,17 @@extern const u8 _binary_fips140_ko_start[];extern const u8 _binary_fips140_ko_end[]; +extern const u8 _binary_fips140_hmac_start[]; +extern const u8 _binary_fips140_hmac_end[]; +const u8 *_binary_crypto_ko_start;EXPORT_SYMBOL_GPL (_binary_crypto_ko_start);const u8 *_binary_crypto_ko_end;EXPORT_SYMBOL_GPL (_binary_crypto_ko_end); +const u8 *_binary_crypto_hmac_start; +EXPORT_SYMBOL_GPL (_binary_crypto_hmac_start); +const u8 *_binary_crypto_hmac_end; +EXPORT_SYMBOL_GPL (_binary_crypto_hmac_end);/* Function to load crypto module from memory */ extern int load_crypto_module_mem (const char *mem,size_t size);
@@ -26,6 +33,8 @@ static void load_prepare(void) { _binary_crypto_ko_start = _binary_fips140_ko_start; _binary_crypto_ko_end = _binary_fips140_ko_end; + _binary_crypto_hmac_start = _binary_fips140_hmac_start; + _binary_crypto_hmac_end = _binary_fips140_hmac_end; }static int fips_loader_init (void )
scripts/Makefile.modfinal +17 -1
@@ -69,12 +69,28 @@ ifeq ($(crypto-module-gen),1) +$(call if_changed,ld_ko_o) else %.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE - +$(call if_changed_except,ld_ko_o,$(objtree)/vmlinux) + +$(call if_changed_except,ld_ko_o_and_cp_extmod,$(objtree)/vmlinux) ifdef CONFIG_DEBUG_INFO_BTF_MODULES +$(if $(newer-prereqs),$(call cmd,btf_ko)) endif +$(call cmd,check_tracepoint) endif + +fips140_build = . +ifeq ($(CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE),y) +fips140_build = fips140_build +endif + +quiet_cmd_ld_ko_o_and_cp_extmod = LD [M] $@ + cmd_ld_ko_o_and_cp_extmod = \ + $(LD) -r $(KBUILD_LDFLAGS) \ + $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE) \ + -T $(objtree)/scripts/module.lds -o $@ $(filter %.o, $^); \ + if [ "$(CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE)" = "y" ] && \ + [ -f "$(fips140_build)/$@" ]; then \ + echo " CP [M] $@"; \ + cp "$(fips140_build)/$@" "$@"; \ + fi else %.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE +$(call if_changed_except,ld_ko_o,$(objtree)/vmlinux)
scripts/Makefile.vmlinux +5 -1
@@ -67,8 +67,12 @@ cmd_link_vmlinux = \ $< "$(LD)" "$(KBUILD_LDFLAGS)" "$(LDFLAGS_vmlinux)" "$@"; \ $(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true) +ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +fips140-deps := crypto/fips140/fips140-embedded.o crypto/fips140/fips140-digest.o +endif + targets += vmlinux.unstripped .vmlinux.export.o -vmlinux.unstripped: scripts/link-vmlinux.sh vmlinux.o .vmlinux.export.o $(KBUILD_LDS) FORCE +vmlinux.unstripped: scripts/link-vmlinux.sh vmlinux.o .vmlinux.export.o $(KBUILD_LDS) $(fips140-deps) FORCE +$(call if_changed_dep,link_vmlinux) ifdef CONFIG_DEBUG_INFO_BTF vmlinux.unstripped: $(RESOLVE_BTFIDS) $(srctree)/scripts/gen-btf.sh
scripts/link-vmlinux.sh +5 -0
@@ -75,6 +75,11 @@ vmlinux_link() fi objs="${objs} .vmlinux.export.o" + + if is_enabled CONFIG_CRYPTO_FIPS140_EXTMOD; then + objs="${objs} crypto/fips140/fips140-embedded.o crypto/fips140/fips140-digest.o" + fi + objs="${objs} init/version-timestamp.o" if [ "${SRCARCH}" = "um" ]; then
Signature · 1 line
2.47.3
PATCH v2 10/19 module: skip modversion checks for crypto modules
In reply to Jay WangThe standalone crypto module feature allows loading pre-built crypto modules from an external build to preserve FIPS certification across kernel updates. Since these externally built modules have different modversion CRCs than the running kernel, the module_layout and per-symbol version checks will fail.
Add a flags field to struct load_info and bypass check_version() and check_modstruct_version() for crypto modules. For fips140.ko loaded from embedded kernel memory, the MODULE_INIT_CRYPTO_FROM_MEM flag is set by the loader. For individual crypto algorithm modules (e.g., authenc.ko, ccm.ko) built with the crypto-objs-m rule, a .fips140_crypto_marker ELF section is detected during early_mod_check() and the MODULE_INIT_CRYPTO_OBJS_M flag is set accordingly.
--- include/uapi/linux/module.h | 1 + kernel/module/internal.h | 1 + kernel/module/main.c | 17 +++++++++++++++++ kernel/module/version.c | 9 +++++++++ 4 files changed, 28 insertions(+)
include/uapi/linux/module.h +1 -0
@@ -10,6 +10,7 @@#ifdef __KERNEL__/* Internal flags */ #define MODULE_INIT_CRYPTO_FROM_MEM (1 << 8) +#define MODULE_INIT_CRYPTO_OBJS_M (1 << 9)#endif #endif /* _UAPI_LINUX_MODULE_H */
kernel/module/internal.h +1 -0
@@ -69,6 +69,7 @@ struct load_info {char *secstrings, *strtab;unsigned long symoffs, stroffs, init_typeoffs, core_typeoffs;bool sig_ok; +int flags;#ifdef CONFIG_KALLSYMS unsigned long mod_kallsyms_init_off;#endif
kernel/module/main.c +17 -0
@@ -3446,6 +3446,22 @@ static int early_mod_check(struct load_info *info, int flags)if (err)return err; +#ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +/* Detect crypto-objs-m modules by .fips140_crypto_marker section */ +if (!(info->flags &MODULE_INIT_CRYPTO_FROM_MEM )) { +unsigned int i; + +for (i =1 ; i < info->hdr->e_shnum; i++) { +const char *sname = info->secstrings + info->sechdrs[i].sh_name; + +if (strcmp (sname,".fips140_crypto_marker" ) ==0 ) { + info->flags |=MODULE_INIT_CRYPTO_OBJS_M ; +break ; + } + } + } +#endif +/* Check module struct version now, before we try to use module. */ if (!check_modstruct_version (info, info->mod))return -ENOEXEC ;
@@ -3678,6 +3694,7 @@ int load_crypto_module_mem(const char *mem, size_t size) } info.sig_ok = true; + info.flags =MODULE_INIT_CRYPTO_FROM_MEM ; info.hdr = (Elf_Ehdr *) mem; info.len = size;
kernel/module/version.c +9 -0
@@ -8,6 +8,7 @@#include <linux/module.h> #include <linux/string.h> #include <linux/printk.h> +#include <uapi/linux/module.h> #include "internal.h" int check_version(const struct load_info *info,
@@ -21,6 +22,10 @@ int check_version(const struct load_info *info,struct modversion_info *versions;struct modversion_info_ext version_ext; +/* Skip version checks for FIPS crypto modules */ +if (info->flags & (MODULE_INIT_CRYPTO_FROM_MEM |MODULE_INIT_CRYPTO_OBJS_M )) +return 1 ; +/* Exporting module didn't supply crcs? OK, we're already tainted. */ if (!crc)return 1 ;
@@ -81,6 +86,10 @@ int check_modstruct_version(const struct load_info *info, };bool have_symbol; +/* Skip module_layout version check for FIPS crypto modules */ +if (info->flags & (MODULE_INIT_CRYPTO_FROM_MEM |MODULE_INIT_CRYPTO_OBJS_M )) +return 1 ; +/* * Since this should be found in kernel (which can't be removed), no * locking is necessary. Regardless use a RCU read section to keep
Signature · 1 line
2.47.3
PATCH v2 11/19 build: add CONFIG_DEBUG_INFO_BTF_MODULES support for the standalone crypto kernel module
In reply to Jay WangWhen CONFIG_DEBUG_INFO_BTF_MODULES=y, there are two problems for the standalone crypto kernel module.
First, it requires a make dependency .ko: vmlinux because it takes vmlinux as input to generate BTF info for the module, and inserts such info into the .ko module binary as a dedicated ELF section. This can cause an unwanted circular make rule dependency fips140.ko:vmlinux because fips140.ko is already embedded into vmlinux.
To address this issue, we reuse the same script to explicitly generate fips140.ko's BTF info in the vmlinux generation process to avoid the circular make dependency. We link vmlinux first, then use it to generate fips140.ko's BTF info, and then embed the fips140.ko and its BTF info back with vmlinux by re-linking. Since the fips140.ko's BTF info is embedded as data only into vmlinux, the BTF info generated using the first linked vmlinux will be the same as if using the latest vmlinux.
Second, CONFIG_DEBUG_INFO_BTF_MODULES=y will insert BTF info into fips140.ko binary, which means the previously generated module signature on "fips140.ko" binary becomes invalid, thus needing regeneration.
To avoid this issue, we don't re-insert module's BTF info into fips140.ko binary (as normally done), but keep such info as a separate file, and embed into vmlinux as separate ELF section. By doing this, the fips140.ko binary remains unchanged while its latest up-to-date BTF info is available to kernel.
--- arch/arm64/kernel/vmlinux.lds.S | 8 ++++++++ arch/x86/kernel/vmlinux.lds.S | 8 ++++++++ crypto/fips140/Makefile | 2 +- crypto/fips140/fips140-loader.c | 12 ++++++++++++ kernel/bpf/btf.c | 20 ++++++++++++++++++++ kernel/module/main.c | 27 +++++++++++++++++++++++++++ scripts/Makefile.vmlinux | 28 ++++++++++++++++++++++++++++ scripts/link-vmlinux.sh | 3 +++ 8 files changed, 107 insertions(+), 1 deletion(-)
arch/arm64/kernel/vmlinux.lds.S +8 -0
@@ -215,6 +215,14 @@ SECTIONS KEEP(*(.fips140_digest)) _binary_fips140_hmac_end = .; } +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES + .fips140_btf : { + . = ALIGN(8); + __start_fips140_btf = .; + KEEP(*(.fips140_btf)) + __stop_fips140_btf = .; + } +#endif #endif HYPERVISOR_RODATA_SECTIONS
arch/x86/kernel/vmlinux.lds.S +8 -0
@@ -185,6 +185,14 @@ SECTIONS KEEP(*(.fips140_digest)) _binary_fips140_hmac_end = .; } +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES + .fips140_btf : AT(ADDR(.fips140_btf) - LOAD_OFFSET) { + . = ALIGN(8); + __start_fips140_btf = .; + KEEP(*(.fips140_btf)) + __stop_fips140_btf = .; + } +#endif #endif /* Data */
crypto/fips140/Makefile +1 -1
@@ -7,4 +7,4 @@ obj-y += fips140-loader.o CFLAGS_fips140-fn-redirect.o += -I$(obj) CFLAGS_fips140-module.o += -DFIPS140_CORE -clean-files:= .fips140.order .fips140.symvers .fips140-fn-redirect.h .fips140.exported .fips140.hmac +clean-files:= .fips140.order .fips140.symvers .fips140-fn-redirect.h .fips140.exported .fips140.hmac .fips140.ko.btf
crypto/fips140/fips140-loader.c +12 -0
@@ -26,6 +26,13 @@ EXPORT_SYMBOL_GPL(_binary_crypto_hmac_start);const u8 *_binary_crypto_hmac_end;EXPORT_SYMBOL_GPL (_binary_crypto_hmac_end); +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES +extern const u8 __start_fips140_btf[]; +extern const u8 __stop_fips140_btf[]; +const u8 *__start_crypto_btf; +const u8 *__stop_crypto_btf; +#endif +/* Function to load crypto module from memory */ extern int load_crypto_module_mem (const char *mem,size_t size);
@@ -35,6 +42,11 @@ static void load_prepare(void) _binary_crypto_ko_end = _binary_fips140_ko_end; _binary_crypto_hmac_start = _binary_fips140_hmac_start; _binary_crypto_hmac_end = _binary_fips140_hmac_end; + +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES + __start_crypto_btf = __start_fips140_btf; + __stop_crypto_btf = __stop_fips140_btf; +#endif }static int fips_loader_init (void )
kernel/bpf/btf.c +20 -0
@@ -8474,6 +8474,26 @@ static int __init btf_module_init(void) return 0; } +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && defined(CONFIG_DEBUG_INFO_BTF_MODULES ) +/* Handle deferred BTF registration for FIPS140 loaded before btf_kobj exists */ +struct module *fips140_deferred_mod =NULL ; + +static int __init register_deferred_fips140_btf (void ) +{ +if (fips140_deferred_mod && btf_kobj) { +/* Manually trigger BTF registration for FIPS140 */ +btf_module_notify (NULL ,MODULE_STATE_COMING , fips140_deferred_mod); + fips140_deferred_mod =NULL ; +pr_info ("FIPS140 BTF registration completed\n" ); + }else { +pr_info ("FIPS140 BTF registration skipped: deferred_mod=%p, btf_kobj=%p\n" , + fips140_deferred_mod, btf_kobj); + } +return 0 ; +} +late_initcall (register_deferred_fips140_btf); +#endif +fs_initcall (btf_module_init);#endif /* CONFIG_DEBUG_INFO_BTF_MODULES */
kernel/module/main.c +27 -0
@@ -2715,6 +2715,23 @@ static int find_module_sections(struct module *mod, struct load_info *info) mod->btf_data =any_section_objs (info,".BTF" ,1 , &mod->btf_data_size); mod->btf_base_data =any_section_objs (info,".BTF.base" ,1 , &mod->btf_base_data_size); + +#ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +/* Inject embedded BTF for FIPS140 module */ +if (!mod->btf_data && (info->flags &MODULE_INIT_CRYPTO_FROM_MEM )) { +extern u8 *__start_crypto_btf; +extern u8 *__stop_crypto_btf; +size_t btf_size = __stop_crypto_btf - __start_crypto_btf; + +pr_info ("FIPS140: Attempting BTF injection, btf_size=%zu\n" , btf_size); + +if (btf_size >0 ) { + mod->btf_data = __start_crypto_btf; + mod->btf_data_size = btf_size; +pr_info ("FIPS140: Injected embedded BTF data, size %zu\n" , btf_size); + } + } +#endif #endif #ifdef CONFIG_JUMP_LABEL mod->jump_entries = section_objs(info, "__jump_table",
@@ -3403,6 +3420,16 @@ static int prepare_coming_module(struct module *mod) err =blocking_notifier_call_chain_robust (&module_notify_list,MODULE_STATE_COMING ,MODULE_STATE_GOING , mod); err =notifier_to_errno (err); +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && defined(CONFIG_DEBUG_INFO_BTF_MODULES ) +/* Since fips140 module is loaded too early when BTF subsystem is not ready, +* record this module for later BTF registration processing */ +if (!strcmp (mod->name,"fips140" )) { +pr_info ("FIPS140 BTF MODULE_STATE_COMING: processing BTF registration\n" ); +extern struct module *fips140_deferred_mod; + fips140_deferred_mod = mod;/* Store for later reference */ + } +#endif +if (err)klp_module_going (mod);
scripts/Makefile.vmlinux +28 -0
@@ -62,10 +62,38 @@ endif ARCH_POSTLINK := $(wildcard $(srctree)/arch/$(SRCARCH)/Makefile.postlink) +fips140_build = . +ifeq ($(CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE),y) +fips140_build = fips140_build +endif + # Final link of vmlinux with optional arch pass after final link cmd_link_vmlinux = \ $< "$(LD)" "$(KBUILD_LDFLAGS)" "$(LDFLAGS_vmlinux)" "$@"; \ $(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true) +ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +ifdef CONFIG_DEBUG_INFO_BTF_MODULES + cmd_link_vmlinux += ; \ + cp $(fips140_build)/crypto/fips140/fips140.ko crypto/fips140/fips140.ko.tmp; \ + LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $@ crypto/fips140/fips140.ko.tmp; \ + $(RESOLVE_BTFIDS) -b $@ crypto/fips140/fips140.ko.tmp; \ + $(OBJCOPY) --dump-section=.BTF=crypto/fips140/.fips140.ko.btf crypto/fips140/fips140.ko.tmp; \ + cp crypto/fips140/.fips140.ko.btf crypto/fips140/.fips140.ko.btf.first; \ + rm -f crypto/fips140/fips140.ko.tmp; \ + $(LD) -r -b binary -o crypto/fips140/fips140_btf.o crypto/fips140/.fips140.ko.btf; \ + $(OBJCOPY) --rename-section .data=.fips140_btf crypto/fips140/fips140_btf.o; \ + rm -f $@; \ + FIPS140_BTF_RELINK=1 $< "$(LD)" "$(KBUILD_LDFLAGS)" "$(LDFLAGS_vmlinux)" "$@"; \ + cp $(fips140_build)/crypto/fips140/fips140.ko crypto/fips140/fips140.ko.tmp2; \ + LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $@ crypto/fips140/fips140.ko.tmp2; \ + $(RESOLVE_BTFIDS) -b $@ crypto/fips140/fips140.ko.tmp2; \ + $(OBJCOPY) --dump-section=.BTF=crypto/fips140/.fips140.ko.btf.second crypto/fips140/fips140.ko.tmp2; \ + rm -f crypto/fips140/fips140.ko.tmp2; \ + diff crypto/fips140/.fips140.ko.btf.first crypto/fips140/.fips140.ko.btf.second >/dev/null || echo "Module BTF differs"; \ + rm -f crypto/fips140/.fips140.ko.btf.first crypto/fips140/.fips140.ko.btf.second; \ + $(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true) +endif +endif ifdef CONFIG_CRYPTO_FIPS140_EXTMOD fips140-deps := crypto/fips140/fips140-embedded.o crypto/fips140/fips140-digest.o
scripts/link-vmlinux.sh +3 -0
@@ -78,6 +78,9 @@ vmlinux_link() if is_enabled CONFIG_CRYPTO_FIPS140_EXTMOD; then objs="${objs} crypto/fips140/fips140-embedded.o crypto/fips140/fips140-digest.o" + if is_enabled CONFIG_DEBUG_INFO_BTF_MODULES && [ -n "${FIPS140_BTF_RELINK}" ] && [ -f crypto/fips140/fips140_btf.o ]; then + objs="${objs} crypto/fips140/fips140_btf.o" + fi fi objs="${objs} init/version-timestamp.o"
Signature · 1 line
2.47.3
PATCH v2 12/19 Allow selective crypto module loading at boot based on FIPS mode
In reply to Jay WangIntroduce CONFIG_CRYPTO_FIPS140_DUAL_VERSION to enable dual crypto module versions within a single kernel build, allowing boot-time selection based on FIPS mode status.
This configuration allows FIPS mode to use pre-compiled certified crypto modules from external source, while regular mode uses freshly built kernel crypto implementation for optimal performance and latest security features.
The implementation embeds both certified and non-certified fips140.ko modules in vmlinux and adds new linker sections (.nonfips140_embedded, .nonfips140_btf) for non-FIPS crypto module storage. It modifies fips140-loader.c to select appropriate module at boot time based on fips_enabled flag, updates build system to generate and embed both module versions, and includes BTF support for both module variants when CONFIG_DEBUG_INFO_BTF_MODULES is enabled.
For modular crypto algorithms (e.g., aes.ko), they are not automatically duplicated. They should either be built-in to fips140.ko for automatic duplication, or require userspace utilities like modprobe to handle proper isolation between FIPS and non-FIPS modular crypto implementations.
--- Makefile | 15 +++++++++++++++ arch/arm64/kernel/vmlinux.lds.S | 16 ++++++++++++++++ arch/x86/kernel/vmlinux.lds.S | 16 ++++++++++++++++ crypto/fips140/Kconfig | 24 ++++++++++++++++++++++++ crypto/fips140/Makefile | 5 ++++- crypto/fips140/fips140-loader.c | 26 ++++++++++++++++++++++++++ scripts/Makefile.vmlinux | 29 +++++++++++++++++++++++++++-- scripts/link-vmlinux.sh | 6 ++++++ 8 files changed, 134 insertions(+), 3 deletions(-)
Makefile +15 -0
@@ -1318,9 +1318,11 @@ quiet_cmd_gen_fips140_exported = awk '{print "0x00000000\t" $$1 "\tcrypto/fips140/fips140\tEXPORT_SYMBOL_GPL\t"}' > $@ \ $(fips140_cp_exported) +ifndef CONFIG_CRYPTO_FIPS140_DUAL_VERSION ifeq ($(CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE),y) fips140_cp_exported = ; cp "$(fips140_build)/crypto/fips140/.fips140.exported" crypto/fips140/.fips140.exported endif +endif crypto/fips140/.fips140.exported: crypto/fips140/fips140.o FORCE $(call if_changed,gen_fips140_exported)
@@ -1367,11 +1369,22 @@ vmlinux: private _LDFLAGS_vmlinux := $(LDFLAGS_vmlinux) vmlinux: export LDFLAGS_vmlinux = $(_LDFLAGS_vmlinux) ifdef CONFIG_CRYPTO_FIPS140_EXTMOD vmlinux: crypto/fips140/fips140-embedded.o crypto/fips140/fips140-digest.o +ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION +vmlinux: crypto/fips140/nonfips140-embedded.o +endif crypto/fips140/fips140-embedded.o: fips140-ready @echo " LD $@" @$(LD) -r -b binary -o $@ $(fips140_build)/crypto/fips140/fips140.ko @$(OBJCOPY) --rename-section .data=.fips140_module_data $@ +ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION +crypto/fips140/nonfips140-embedded.o: fips140-ready + @echo " LD $@" + @$(LD) -r -b binary -o $@ crypto/fips140/fips140.ko + @$(OBJCOPY) --rename-section .data=.nonfips140_module_data \ + --prefix-symbols nonfips140_ $@ +endif + crypto/fips140/.fips140.hmac: crypto/fips140/fips140-embedded.o @echo " HMAC $@" @hmac_key=$$(awk -F'"' '/^CONFIG_CRYPTO_FIPS140_HMAC_KEY=/{print $$2}' .config); \
@@ -1389,9 +1402,11 @@ fips140-ready: crypto/fips140/fips140.o crypto/fips140/.fips140.order crypto/fip ifneq ($(KBUILD_MODPOST_NOFINAL),1) $(Q)$(MAKE) KBUILD_MODULES=y crypto-module-gen=1 -f $(srctree)/scripts/Makefile.modfinal endif +ifndef CONFIG_CRYPTO_FIPS140_DUAL_VERSION ifeq ($(CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE),y) cp "$(fips140_build)/crypto/fips140/fips140.ko" crypto/fips140/fips140.ko; endif +endif # Generate fips140.o from crypto-module.a files crypto/fips140/fips140.o: crypto-module.a FORCE
arch/arm64/kernel/vmlinux.lds.S +16 -0
@@ -223,6 +223,22 @@ SECTIONS __stop_fips140_btf = .; } #endif +#ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION + .nonfips140_embedded : { + . = ALIGN(8); + _binary_nonfips140_ko_start = .; + KEEP(*(.nonfips140_module_data)) + _binary_nonfips140_ko_end = .; + } +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES + .nonfips140_btf : { + . = ALIGN(8); + __start_nonfips140_btf = .; + KEEP(*(.nonfips140_btf)) + __stop_nonfips140_btf = .; + } +#endif +#endif #endif HYPERVISOR_RODATA_SECTIONS
arch/x86/kernel/vmlinux.lds.S +16 -0
@@ -193,6 +193,22 @@ SECTIONS __stop_fips140_btf = .; } #endif +#ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION + .nonfips140_embedded : AT(ADDR(.nonfips140_embedded) - LOAD_OFFSET) { + . = ALIGN(8); + _binary_nonfips140_ko_start = .; + KEEP(*(.nonfips140_module_data)) + _binary_nonfips140_ko_end = .; + } +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES + .nonfips140_btf : AT(ADDR(.nonfips140_btf) - LOAD_OFFSET) { + . = ALIGN(8); + __start_nonfips140_btf = .; + KEEP(*(.nonfips140_btf)) + __stop_nonfips140_btf = .; + } +#endif +#endif #endif /* Data */
crypto/fips140/Kconfig +24 -0
@@ -42,3 +42,27 @@ config CRYPTO_FIPS140_EXTMOD_SOURCE - fips140_build/crypto/sha256.ko If unsure, say N. +config CRYPTO_FIPS140_DUAL_VERSION + bool "Enable dual crypto versions for FIPS and regular modes" + depends on CRYPTO_FIPS140_EXTMOD && CRYPTO_FIPS140_EXTMOD_SOURCE + default n + help + Enable keeping two crypto module versions in the same kernel build + for boot-time switching based on FIPS mode status. This allows: + - Non-FIPS users: Get latest crypto algorithms built from current + kernel sources for optimal performance and security features + - FIPS users: Get pre-compiled certified crypto modules that have + undergone formal validation and certification processes + + When enabled: + + For core fips140.ko: + - FIPS mode: Uses certified module from CRYPTO_FIPS140_EXTMOD_SOURCE + - Regular mode: Uses freshly built kernel crypto implementation + + For modular algorithms (e.g., aes.ko), they are not duplicated + automatically. Either make them built-in to be included into + fips140.ko for automatic duplication, or require OS utilities such + as `modprobe` to correctly isolate modular cryptos in filesystems. + + If unsure, say N.
crypto/fips140/Makefile +4 -1
@@ -7,4 +7,7 @@ obj-y += fips140-loader.o CFLAGS_fips140-fn-redirect.o += -I$(obj) CFLAGS_fips140-module.o += -DFIPS140_CORE -clean-files:= .fips140.order .fips140.symvers .fips140-fn-redirect.h .fips140.exported .fips140.hmac .fips140.ko.btf +clean-files:= .fips140.order .fips140.symvers .fips140-fn-redirect.h .fips140.exported .fips140.hmac .fips140.ko.btf +ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION +clean-files += .nonfips140.ko.btf +endif
crypto/fips140/fips140-loader.c +26 -0
@@ -11,12 +11,20 @@#include <linux/elf.h> #include <linux/kthread.h> #include <linux/wait.h> +#include <linux/fips.h> extern const u8 _binary_fips140_ko_start[];extern const u8 _binary_fips140_ko_end[];extern const u8 _binary_fips140_hmac_start[];extern const u8 _binary_fips140_hmac_end[]; +#ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION +/* For non-FIPS mode: no module signature/HMAC is required, +* so only include binary start/end address without module sig address */ +extern const u8 _binary_nonfips140_ko_start[]; +extern const u8 _binary_nonfips140_ko_end[]; +#endif +const u8 *_binary_crypto_ko_start;EXPORT_SYMBOL_GPL (_binary_crypto_ko_start);const u8 *_binary_crypto_ko_end;
@@ -29,6 +37,10 @@ EXPORT_SYMBOL_GPL(_binary_crypto_hmac_end);#ifdef CONFIG_DEBUG_INFO_BTF_MODULES extern const u8 __start_fips140_btf[];extern const u8 __stop_fips140_btf[]; +#ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION +extern const u8 __start_nonfips140_btf[]; +extern const u8 __stop_nonfips140_btf[]; +#endif const u8 *__start_crypto_btf;const u8 *__stop_crypto_btf;#endif
@@ -47,6 +59,20 @@ static void load_prepare(void) __start_crypto_btf = __start_fips140_btf; __stop_crypto_btf = __stop_fips140_btf;#endif + +#ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION +if (!fips_enabled) { + _binary_crypto_ko_start = _binary_nonfips140_ko_start; + _binary_crypto_ko_end = _binary_nonfips140_ko_end; + _binary_crypto_hmac_start =NULL ; + _binary_crypto_hmac_end =NULL ; + +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES + __start_crypto_btf = __start_nonfips140_btf; + __stop_crypto_btf = __stop_nonfips140_btf; +#endif + } +#endif }static int fips_loader_init (void )
scripts/Makefile.vmlinux +27 -2
@@ -81,7 +81,18 @@ ifdef CONFIG_DEBUG_INFO_BTF_MODULES cp crypto/fips140/.fips140.ko.btf crypto/fips140/.fips140.ko.btf.first; \ rm -f crypto/fips140/fips140.ko.tmp; \ $(LD) -r -b binary -o crypto/fips140/fips140_btf.o crypto/fips140/.fips140.ko.btf; \ - $(OBJCOPY) --rename-section .data=.fips140_btf crypto/fips140/fips140_btf.o; \ + $(OBJCOPY) --rename-section .data=.fips140_btf crypto/fips140/fips140_btf.o +ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION + cmd_link_vmlinux += ; \ + cp crypto/fips140/fips140.ko crypto/fips140/nonfips140.ko.tmp; \ + LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $@ crypto/fips140/nonfips140.ko.tmp; \ + $(RESOLVE_BTFIDS) -b $@ crypto/fips140/nonfips140.ko.tmp; \ + $(OBJCOPY) --dump-section=.BTF=crypto/fips140/.nonfips140.ko.btf crypto/fips140/nonfips140.ko.tmp; \ + rm -f crypto/fips140/nonfips140.ko.tmp; \ + $(LD) -r -b binary -o crypto/fips140/nonfips140_btf.o crypto/fips140/.nonfips140.ko.btf; \ + $(OBJCOPY) --rename-section .data=.nonfips140_btf --prefix-symbols nonfips140_ crypto/fips140/nonfips140_btf.o +endif + cmd_link_vmlinux += ; \ rm -f $@; \ FIPS140_BTF_RELINK=1 $< "$(LD)" "$(KBUILD_LDFLAGS)" "$(LDFLAGS_vmlinux)" "$@"; \ cp $(fips140_build)/crypto/fips140/fips140.ko crypto/fips140/fips140.ko.tmp2; \
@@ -90,13 +101,27 @@ ifdef CONFIG_DEBUG_INFO_BTF_MODULES $(OBJCOPY) --dump-section=.BTF=crypto/fips140/.fips140.ko.btf.second crypto/fips140/fips140.ko.tmp2; \ rm -f crypto/fips140/fips140.ko.tmp2; \ diff crypto/fips140/.fips140.ko.btf.first crypto/fips140/.fips140.ko.btf.second >/dev/null || echo "Module BTF differs"; \ - rm -f crypto/fips140/.fips140.ko.btf.first crypto/fips140/.fips140.ko.btf.second; \ + rm -f crypto/fips140/.fips140.ko.btf.first crypto/fips140/.fips140.ko.btf.second +ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION + cmd_link_vmlinux += ; \ + cp crypto/fips140/fips140.ko crypto/fips140/nonfips140.ko.tmp2; \ + LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $@ crypto/fips140/nonfips140.ko.tmp2; \ + $(RESOLVE_BTFIDS) -b $@ crypto/fips140/nonfips140.ko.tmp2; \ + $(OBJCOPY) --dump-section=.BTF=crypto/fips140/.nonfips140.ko.btf.second crypto/fips140/nonfips140.ko.tmp2; \ + rm -f crypto/fips140/nonfips140.ko.tmp2; \ + diff crypto/fips140/.nonfips140.ko.btf crypto/fips140/.nonfips140.ko.btf.second >/dev/null || echo "Nonfips140 Module BTF differs"; \ + rm -f crypto/fips140/.nonfips140.ko.btf.second +endif + cmd_link_vmlinux += ; \ $(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) $@, true) endif endif ifdef CONFIG_CRYPTO_FIPS140_EXTMOD fips140-deps := crypto/fips140/fips140-embedded.o crypto/fips140/fips140-digest.o +ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION +fips140-deps += crypto/fips140/nonfips140-embedded.o +endif endif targets += vmlinux.unstripped .vmlinux.export.o
scripts/link-vmlinux.sh +6 -0
@@ -78,8 +78,14 @@ vmlinux_link() if is_enabled CONFIG_CRYPTO_FIPS140_EXTMOD; then objs="${objs} crypto/fips140/fips140-embedded.o crypto/fips140/fips140-digest.o" + if is_enabled CONFIG_CRYPTO_FIPS140_DUAL_VERSION; then + objs="${objs} crypto/fips140/nonfips140-embedded.o" + fi if is_enabled CONFIG_DEBUG_INFO_BTF_MODULES && [ -n "${FIPS140_BTF_RELINK}" ] && [ -f crypto/fips140/fips140_btf.o ]; then objs="${objs} crypto/fips140/fips140_btf.o" + if is_enabled CONFIG_CRYPTO_FIPS140_DUAL_VERSION && [ -f crypto/fips140/nonfips140_btf.o ]; then + objs="${objs} crypto/fips140/nonfips140_btf.o" + fi fi fi
Signature · 1 line
2.47.3
PATCH v2 13/19 Execute crypto initcalls during module initialization
In reply to Jay WangAfter moving builtin cryptos into the standalone crypto module, to ensure all such cryptos are properly initialized as they were builtin, the initcalls of the cryptos should be executed at proper booting time.
To do so, this patch adds run_initcalls() function to execute the initialization calls of the collected cryptos right after the module is loaded. The function iterates through initcalls in the __fips_initcalls section (which stores all function addresses of the collected crypto initcalls) and executes them.
A key consideration is to ensure the crypto initcalls are executed in proper order, for instance, some crypto initcalls are ought to execute at late_initcall boot time level while some should be executed at module_init boot time level. To do so, this patch enables coordination between kernel and the crypto module initialization to preserve proper execution order by hijacking the kernel initialization function do_initcall_level with added synchronization helpers.
--- crypto/fips140/fips140-loader.c | 93 +++++++++++++++++++++++++++++++++ crypto/fips140/fips140-module.c | 67 ++++++++++++++++++++++++ crypto/fips140/fips140-module.h | 12 +++++ include/linux/init.h | 10 ++++ init/main.c | 4 ++ 5 files changed, 186 insertions(+)
crypto/fips140/fips140-loader.c +93 -0
@@ -100,3 +100,96 @@ static int fips_loader_init(void)vfree (vmalloc_mem);// Free after successful module loading return ret; } + +/* FIPS140 synchronization between kernel and module +* +* Synchronization levels map kernel initcall levels to FIPS module levels: +* - Level 0: subsys_initcall (kernel init level 4) - Basic subsystem initialization +* - Level 1: device_initcall (kernel init level 6) - Device driver initialization +* - Level 2: late_initcall (kernel init level 7) - Late system initialization +* +* The kernel marks each level complete and waits for the FIPS module to +* complete the corresponding level before proceeding to ensure proper +* initialization ordering between kernel crypto and FIPS module. +*/ +atomic_t fips140_kernel_level_complete =ATOMIC_INIT (0 ); +atomic_t fips140_module_level_complete =ATOMIC_INIT (0 ); + +/* Wait queues for efficient synchronization */ +DECLARE_WAIT_QUEUE_HEAD (fips140_kernel_wq); +DECLARE_WAIT_QUEUE_HEAD (fips140_module_wq); + +void fips140_mark_kernel_level_complete (int level) +{ +atomic_or (1 << level, &fips140_kernel_level_complete); +wake_up (&fips140_kernel_wq); +} + +bool fips140_is_kernel_level_complete (int level) +{ +return atomic_read (&fips140_kernel_level_complete) & (1 << level); +} + +bool fips140_is_module_level_complete (int level) +{ +return atomic_read (&fips140_module_level_complete) & (1 << level); +} + +void fips140_mark_module_level_complete (int level) +{ +atomic_or (1 << level, &fips140_module_level_complete); +wake_up (&fips140_module_wq); +} + +static int fips140_sync_thread (void *data) +{ +pr_info ("FIPS 140: starting sync thread\n" ); + +/* Call FIPS loader explicitly */ +int ret =fips_loader_init (); +if (ret) +panic ("FIPS 140: loader initialization failed: %d\n" , ret); + +pr_info ("FIPS 140: sync thread finished\n" ); +return 0 ; +} + +void __init start_fips140_loader (void ) +{ +struct task_struct *task; + + task =kthread_run (fips140_sync_thread,NULL ,"fips140_sync" ); +if (IS_ERR (task)) { +panic ("FIPS 140: failed to create sync thread\n" ); + } +} + +void __init wait_until_fips140_level_sync (int level) +{ +/* Map kernel initcall levels to FIPS module levels */ +int fips_level =-1 ; +if (level ==3 ) {/* Start FIPS loader thread at arch_initcall_sync level */ +start_fips140_loader (); + fips_level =0 ; + }else if (level ==4 )/* subsys_initcall */ + fips_level =1 ; +else if (level ==6 )/* device_initcall */ + fips_level =2 ; +else if (level ==7 )/* late_initcall */ + fips_level =3 ; + +if (fips_level >=0 ) { +/* Mark kernel level complete and wait for module level completion */ +fips140_mark_kernel_level_complete (fips_level); +wait_event (fips140_module_wq,fips140_is_module_level_complete (fips_level)); + } +} + +EXPORT_SYMBOL (fips140_kernel_level_complete); +EXPORT_SYMBOL (fips140_module_level_complete); +EXPORT_SYMBOL (fips140_kernel_wq); +EXPORT_SYMBOL (fips140_module_wq); +EXPORT_SYMBOL (fips140_mark_kernel_level_complete); +EXPORT_SYMBOL (fips140_is_kernel_level_complete); +EXPORT_SYMBOL (fips140_is_module_level_complete); +EXPORT_SYMBOL (fips140_mark_module_level_complete);
crypto/fips140/fips140-module.c +67 -0
76 lines not on this page · raw message
crypto/fips140/fips140-module.h +12 -0
@@ -10,5 +10,17 @@#include <linux/crypto.h> #include <crypto/algapi.h> #include <linux/init.h> +#include <linux/atomic.h> +#include <linux/wait.h> + +/* FIPS140 synchronization between kernel and module */ +extern atomic_t fips140_kernel_level_complete; +extern atomic_t fips140_module_level_complete; +extern wait_queue_head_t fips140_kernel_wq; + +void fips140_mark_kernel_level_complete (int level); +bool fips140_is_kernel_level_complete (int level); +bool fips140_is_module_level_complete (int level); +void fips140_mark_module_level_complete (int level);#endif /* _CRYPTO_FIPS140_MODULE_H */
include/linux/init.h +10 -0
@@ -392,4 +392,14 @@ void __init parse_early_options(char *cmdline);#define __exit_p (x) NULL#endif +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && !defined(__ASSEMBLY__) +/* FIPS140 synchronization between kernel and module */ +void fips140_mark_kernel_level_complete (int level); +bool fips140_is_kernel_level_complete (int level); +bool fips140_is_module_level_complete (int level); +void fips140_mark_module_level_complete (int level); +void start_fips140_loader (void ); +void wait_until_fips140_level_sync (int level); +#endif /* CONFIG_CRYPTO_FIPS140_EXTMOD && !__ASSEMBLY__ */ +#endif /* _LINUX_INIT_H */
init/main.c +4 -0
@@ -1452,6 +1452,10 @@ static void __init do_initcall_level(int level, char *command_line)do_trace_initcall_level (initcall_level_names[level]);for (fn = initcall_levels[level]; fn < initcall_levels[level+1 ]; fn++)do_one_initcall (initcall_from_entry (fn)); + +#ifdef CONFIG_CRYPTO_FIPS140_EXTMOD +wait_until_fips140_level_sync (level); +#endif }static void __init do_initcalls (void )
Signature · 1 line
2.47.3
PATCH v2 14/19 crypto/algapi.c: skip crypto_check_module_sig() for the standalone crypto module
In reply to Jay WangFrom: Vegard Nossum <[email protected]>
The signature check in regular module verification process crypto_check_module_sig() is skipped for this standalone crypto module because its signature has already been checked during load (as described in later patch).
--- crypto/algapi.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)
crypto/algapi.c +13 -1
@@ -24,7 +24,19 @@ static LIST_HEAD(crypto_template_list);static inline void crypto_check_module_sig (struct module *mod) { -if (fips_enabled && mod && !module_sig_ok (mod)) +#ifdef FIPS_MODULE +/* +* The FIPS module should ignore its own signature check, as it was +* already been verified elsewhere during loading. +*/ +if (mod ==THIS_MODULE ) +return ; +#else +if (!fips_enabled) +return ; +#endif + +if (mod && !module_sig_ok (mod))panic ("Module %s signature verification failed in FIPS mode\n" ,module_name (mod)); }
Signature · 1 line
2.47.3
PATCH v2 15/19 crypto: fips140: add module integrity self-check
In reply to Jay WangAdd verify_integrity() function to perform HMAC-SHA256 self verification of this standalone crypto module against pre-computed hash. This integrity self-check is required by FIPS 140-3.
This patch is picked from Vegard Nossum <[email protected]> with minor modifications.
--- crypto/fips140/Kconfig | 2 ++ crypto/fips140/fips140-module.c | 52 +++++++++++++++++++++++++++++++++ crypto/fips140/fips140-module.h | 4 +++ 3 files changed, 58 insertions(+)
crypto/fips140/Kconfig +2 -0
@@ -2,6 +2,8 @@ config CRYPTO_FIPS140_EXTMOD bool "FIPS 140 compliant algorithms as a kernel module" depends on CRYPTO && (X86_64 || ARM64) && MODULES select CRYPTO_FIPS + select CRYPTO_SHA256 + select CRYPTO_HMAC help This option enables building a kernel module that contains copies of crypto algorithms that are built in a way that
crypto/fips140/fips140-module.c +52 -0
64 lines not on this page · raw message
crypto/fips140/fips140-module.h +4 -0
@@ -9,9 +9,13 @@#include <linux/module.h> #include <linux/crypto.h> #include <crypto/algapi.h> +#include <crypto/hash.h> +#include <crypto/sha2.h> #include <linux/init.h> +#include <linux/string.h> #include <linux/atomic.h> #include <linux/wait.h> +#include <linux/fips.h> /* FIPS140 synchronization between kernel and module */ extern atomic_t fips140_kernel_level_complete;
Signature · 1 line
2.47.3
PATCH v2 16/19 crypto: convert exported symbols in architecture-independent crypto to pluggable symbols
In reply to Jay WangInclude architecture-independent crypto source files in fips140.ko by converting their Makefile rules from obj-y to crypto-objs-y, and apply the pluggable interface introduced in the earlier patch so that symbols defined in fips140.ko can still be referenced by vmlinux.
For exported function symbols (the majority, ~hundreds), the --wrap linker mechanism automatically redirects all references in vmlinux to trampolines, so no source tree modifications are needed beyond ensuring each function has an EXPORT_SYMBOL. A few functions that were not previously exported now need EXPORT_SYMBOL_GPL added since they move into the module.
For exported variable symbols (a small number), the header declarations are replaced with DECLARE_CRYPTO_VAR() and a macro that redirects accesses through a pointer indirection. The placeholder definitions are added via DEFINE_CRYPTO_VAR_STUB() in fips140-var-redirect.c. The wrapper takes the Kconfig symbol as a parameter so that when a crypto algorithm is already configured as a module (=m), the original declaration remains unchanged.
--- certs/system_keyring.c | 1 + crypto/Makefile | 202 ++++++++++++------------- crypto/asymmetric_keys/Makefile | 16 +- crypto/asymmetric_keys/restrict.c | 2 + crypto/asymmetric_keys/verify_pefile.c | 1 + crypto/async_tx/Makefile | 12 +- crypto/fips140/Makefile | 16 +- crypto/fips140/fips140-var-redirect.c | 78 ++++++++++ crypto/krb5/Makefile | 2 +- include/crypto/cast_common.h | 17 ++- include/crypto/md5.h | 8 +- include/crypto/public_key.h | 7 +- include/crypto/rng.h | 1 + include/crypto/sm4.h | 13 +- include/keys/asymmetric-type.h | 7 +- 15 files changed, 255 insertions(+), 128 deletions(-) create mode 100644 crypto/fips140/fips140-var-redirect.c
certs/system_keyring.c +1 -0
@@ -50,6 +50,7 @@ int restrict_link_by_builtin_trusted(struct key *dest_keyring,return restrict_link_by_signature (dest_keyring, type, payload, builtin_trusted_keys); } +EXPORT_SYMBOL_GPL (restrict_link_by_builtin_trusted);/** * restrict_link_by_digsig_builtin - Restrict digitalSignature key additions by the built-in keyring
crypto/Makefile +101 -101
284 lines not on this page · raw message
crypto/asymmetric_keys/Makefile +8 -8
@@ -3,26 +3,26 @@ # Makefile for asymmetric cryptographic keys # -obj-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o +crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o asymmetric_keys-y := \ asymmetric_type.o \ restrict.o \ signature.o -obj-$(CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE) += public_key.o +crypto-objs-$(CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE) += public_key.o # # X.509 Certificate handling # -obj-$(CONFIG_X509_CERTIFICATE_PARSER) += x509_key_parser.o +crypto-objs-$(CONFIG_X509_CERTIFICATE_PARSER) += x509_key_parser.o x509_key_parser-y := \ x509.asn1.o \ x509_akid.asn1.o \ x509_cert_parser.o \ x509_loader.o \ x509_public_key.o -obj-$(CONFIG_FIPS_SIGNATURE_SELFTEST) += x509_selftest.o +crypto-objs-$(CONFIG_FIPS_SIGNATURE_SELFTEST) += x509_selftest.o x509_selftest-y += selftest.o x509_selftest-$(CONFIG_FIPS_SIGNATURE_SELFTEST_RSA) += selftest_rsa.o x509_selftest-$(CONFIG_FIPS_SIGNATURE_SELFTEST_ECDSA) += selftest_ecdsa.o
@@ -37,7 +37,7 @@ $(obj)/x509_akid.asn1.o: $(obj)/x509_akid.asn1.c $(obj)/x509_akid.asn1.h # # PKCS#8 private key handling # -obj-$(CONFIG_PKCS8_PRIVATE_KEY_PARSER) += pkcs8_key_parser.o +crypto-objs-$(CONFIG_PKCS8_PRIVATE_KEY_PARSER) += pkcs8_key_parser.o pkcs8_key_parser-y := \ pkcs8.asn1.o \ pkcs8_parser.o
@@ -50,7 +50,7 @@ clean-files += pkcs8.asn1.c pkcs8.asn1.h # # PKCS#7 message handling # -obj-$(CONFIG_PKCS7_MESSAGE_PARSER) += pkcs7_message.o +crypto-objs-$(CONFIG_PKCS7_MESSAGE_PARSER) += pkcs7_message.o pkcs7_message-y := \ pkcs7.asn1.o \ pkcs7_parser.o \
@@ -63,14 +63,14 @@ $(obj)/pkcs7.asn1.o: $(obj)/pkcs7.asn1.c $(obj)/pkcs7.asn1.h # # PKCS#7 parser testing key # -obj-$(CONFIG_PKCS7_TEST_KEY) += pkcs7_test_key.o +crypto-objs-$(CONFIG_PKCS7_TEST_KEY) += pkcs7_test_key.o pkcs7_test_key-y := \ pkcs7_key_type.o # # Signed PE binary-wrapped key handling # -obj-$(CONFIG_SIGNED_PE_FILE_VERIFICATION) += verify_signed_pefile.o +crypto-objs-$(CONFIG_SIGNED_PE_FILE_VERIFICATION) += verify_signed_pefile.o verify_signed_pefile-y := \ verify_pefile.o \
crypto/asymmetric_keys/restrict.c +2 -0
@@ -114,6 +114,7 @@ int restrict_link_by_signature(struct key *dest_keyring,key_put (key);return ret; } +EXPORT_SYMBOL_GPL (restrict_link_by_signature);/** * restrict_link_by_ca - Restrict additions to a ring of CA keys
@@ -198,6 +199,7 @@ int restrict_link_by_digsig(struct key *dest_keyring,return restrict_link_by_signature (dest_keyring, type, payload, trust_keyring); } +EXPORT_SYMBOL_GPL (restrict_link_by_digsig);static bool match_either_id (const struct asymmetric_key_id **pair,const struct asymmetric_key_id *single)
crypto/asymmetric_keys/verify_pefile.c +1 -0
@@ -454,3 +454,4 @@ int verify_pefile_signature(const void *pebuf, unsigned pelen,kfree_sensitive (ctx.digest);return ret; } +EXPORT_SYMBOL_GPL (verify_pefile_signature);
crypto/async_tx/Makefile +6 -6
@@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 -obj-$(CONFIG_ASYNC_CORE) += async_tx.o -obj-$(CONFIG_ASYNC_MEMCPY) += async_memcpy.o -obj-$(CONFIG_ASYNC_XOR) += async_xor.o -obj-$(CONFIG_ASYNC_PQ) += async_pq.o -obj-$(CONFIG_ASYNC_RAID6_RECOV) += async_raid6_recov.o -obj-$(CONFIG_ASYNC_RAID6_TEST) += raid6test.o +crypto-objs-$(CONFIG_ASYNC_CORE) += async_tx.o +crypto-objs-$(CONFIG_ASYNC_MEMCPY) += async_memcpy.o +crypto-objs-$(CONFIG_ASYNC_XOR) += async_xor.o +crypto-objs-$(CONFIG_ASYNC_PQ) += async_pq.o +crypto-objs-$(CONFIG_ASYNC_RAID6_RECOV) += async_raid6_recov.o +crypto-objs-$(CONFIG_ASYNC_RAID6_TEST) += raid6test.o
crypto/fips140/Makefile +14 -2
@@ -1,11 +1,23 @@ crypto-objs-y += \ - fips140-module.o + fips140-module.o \ + fips140-var-redirect-fips.o -obj-y += fips140-loader.o +obj-y += fips140-loader.o fips140-var-redirect-main.o CFLAGS_fips140-fn-redirect.o += -I$(obj) CFLAGS_fips140-module.o += -DFIPS140_CORE +CFLAGS_fips140-var-redirect-fips.o += -DFIPS140_CORE + +# Explicit rules to compile same source to different objects +$(obj)/fips140-var-redirect-main.o: $(src)/fips140-var-redirect.c FORCE + $(call if_changed_rule,cc_o_c) + +$(obj)/fips140-var-redirect-fips.o: $(src)/fips140-var-redirect.c FORCE + $(call if_changed_rule,cc_o_c) + +CFLAGS_fips140-var-redirect-main.o += -I$(srctree) +CFLAGS_fips140-var-redirect-fips.o += -I$(srctree) clean-files:= .fips140.order .fips140.symvers .fips140-fn-redirect.h .fips140.exported .fips140.hmac .fips140.ko.btf ifdef CONFIG_CRYPTO_FIPS140_DUAL_VERSION
crypto/fips140/fips140-var-redirect.c new file +78 -0
79 lines not on this page · raw message
crypto/krb5/Makefile +1 -1
@@ -15,4 +15,4 @@ krb5-$(CONFIG_CRYPTO_KRB5_SELFTESTS) += \ selftest.o \ selftest_data.o -obj-$(CONFIG_CRYPTO_KRB5) += krb5.o +crypto-objs-$(CONFIG_CRYPTO_KRB5) += krb5.o
include/crypto/cast_common.h +13 -4
@@ -2,9 +2,18 @@#ifndef _CRYPTO_CAST_COMMON_H#define _CRYPTO_CAST_COMMON_H -extern const u32 cast_s1[256 ]; -extern const u32 cast_s2[256 ]; -extern const u32 cast_s3[256 ]; -extern const u32 cast_s4[256 ]; +#include <crypto/fips140-redirect.h> + +DECLARE_CRYPTO_VAR(CONFIG_CRYPTO_CAST_COMMON, cast_s1, const u32, [256]); +DECLARE_CRYPTO_VAR(CONFIG_CRYPTO_CAST_COMMON, cast_s2, const u32, [256]); +DECLARE_CRYPTO_VAR(CONFIG_CRYPTO_CAST_COMMON, cast_s3, const u32, [256]); +DECLARE_CRYPTO_VAR(CONFIG_CRYPTO_CAST_COMMON, cast_s4, const u32, [256]); + +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && !defined(FIPS_MODULE ) &&IS_BUILTIN (CONFIG_CRYPTO_CAST_COMMON ) +#define cast_s1 (((const u32*)CRYPTO_VAR_NAME(cast_s1))) +#define cast_s2 (((const u32*)CRYPTO_VAR_NAME(cast_s2))) +#define cast_s3 (((const u32*)CRYPTO_VAR_NAME(cast_s3))) +#define cast_s4 (((const u32*)CRYPTO_VAR_NAME(cast_s4))) +#endif #endif
include/crypto/md5.h +7 -1
@@ -20,7 +20,13 @@#define CRYPTO_MD5_STATESIZE \ CRYPTO_HASH_STATESIZE(MD5_STATE_SIZE, MD5_HMAC_BLOCK_SIZE) -extern const u8 md5_zero_message_hash[MD5_DIGEST_SIZE ]; +#include <crypto/fips140-redirect.h> + +DECLARE_CRYPTO_VAR(CONFIG_CRYPTO_MD5, md5_zero_message_hash, const u8, [MD5_DIGEST_SIZE]); + +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && !defined(FIPS_MODULE ) &&IS_BUILTIN (CONFIG_CRYPTO_MD5 ) +#define md5_zero_message_hash (((const u8*)CRYPTO_VAR_NAME(md5_zero_message_hash))) +#endif struct md5_state {u32 hash[MD5_HASH_WORDS ];
include/crypto/public_key.h +6 -1
@@ -13,6 +13,7 @@#include <linux/errno.h> #include <linux/keyctl.h> #include <linux/oid_registry.h> +#include <crypto/fips140-redirect.h> /* * Cryptographic data for the public-key subtype of the asymmetric key type.
@@ -55,7 +56,11 @@ struct public_key_signature {extern void public_key_signature_free (struct public_key_signature *sig); -extern struct asymmetric_key_subtype public_key_subtype; +DECLARE_CRYPTO_VAR(CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE, public_key_subtype, struct asymmetric_key_subtype, ); + +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && !defined(FIPS_MODULE ) &&IS_BUILTIN (CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE ) +#define public_key_subtype (*((struct asymmetric_key_subtype*)CRYPTO_VAR_NAME(public_key_subtype))) +#endif struct key ;struct key_type ;
include/crypto/rng.h +1 -0
@@ -14,6 +14,7 @@#include <linux/crypto.h> #include <linux/fips.h> #include <linux/random.h> +#include <crypto/fips140-redirect.h> struct crypto_rng ;
include/crypto/sm4.h +10 -3
@@ -11,6 +11,7 @@#include <linux/types.h> #include <linux/crypto.h> +#include <crypto/fips140-redirect.h> #define SM4_KEY_SIZE 16#define SM4_BLOCK_SIZE 16
@@ -21,9 +22,15 @@ struct sm4_ctx {u32 rkey_dec[SM4_RKEY_WORDS ]; }; -extern const u32 crypto_sm4_fk[]; -extern const u32 crypto_sm4_ck[]; -extern const u8 crypto_sm4_sbox[]; +DECLARE_CRYPTO_VAR(CONFIG_CRYPTO_SM4, crypto_sm4_fk, const u32, [4]); +DECLARE_CRYPTO_VAR(CONFIG_CRYPTO_SM4, crypto_sm4_ck, const u32, [32]); +DECLARE_CRYPTO_VAR(CONFIG_CRYPTO_SM4, crypto_sm4_sbox, const u8, [256]); + +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && !defined(FIPS_MODULE ) &&IS_BUILTIN (CONFIG_CRYPTO_SM4 ) +#define crypto_sm4_fk (((const u32*)CRYPTO_VAR_NAME(crypto_sm4_fk))) +#define crypto_sm4_ck (((const u32*)CRYPTO_VAR_NAME(crypto_sm4_ck))) +#define crypto_sm4_sbox (((const u8*)CRYPTO_VAR_NAME(crypto_sm4_sbox))) +#endif /** * sm4_expandkey - Expands the SM4 key as described in GB/T 32907-2016
include/keys/asymmetric-type.h +6 -1
@@ -12,8 +12,13 @@#include <linux/key-type.h> #include <linux/verification.h> +#include <crypto/fips140-redirect.h> -extern struct key_type key_type_asymmetric; +DECLARE_CRYPTO_VAR(CONFIG_ASYMMETRIC_KEY_TYPE, key_type_asymmetric, struct key_type, ); + +#if defined(CONFIG_CRYPTO_FIPS140_EXTMOD ) && !defined(FIPS_MODULE ) &&IS_BUILTIN (CONFIG_ASYMMETRIC_KEY_TYPE ) +#define key_type_asymmetric (*((struct key_type*)CRYPTO_VAR_NAME(key_type_asymmetric))) +#endif /* * The key payload is four words. The asymmetric-type key uses them as
Signature · 1 line
2.47.3
PATCH v2 17/19 x86/crypto: convert exported symbols in x86 crypto to pluggable symbols
In reply to Jay WangInclude x86 architecture-specific crypto source files in fips140.ko by converting their Makefile rules from obj-y to crypto-objs-y, and apply the pluggable interface introduced in the earlier patch so that symbols defined in fips140.ko can still be referenced by vmlinux.
For exported function symbols, the --wrap linker mechanism automatically redirects all references in vmlinux to trampolines, so no source tree modifications are needed beyond ensuring each function has an EXPORT_SYMBOL.
For exported variable symbols, an architecture-specific fips140-var-redirect.c is introduced under arch/x86/crypto/fips140/ to hold DEFINE_CRYPTO_VAR_STUB() definitions. This file is compiled twice: once for vmlinux (as the "outlet" providing the placeholder pointers) and once for fips140.ko (as the "plug" populating them with real addresses via the __crypto_var_keys section).
--- arch/x86/crypto/Makefile | 41 ++++++++++--------- arch/x86/crypto/fips140/Makefile | 14 +++++++ .../x86/crypto/fips140/fips140-var-redirect.c | 0 3 files changed, 36 insertions(+), 19 deletions(-) create mode 100644 arch/x86/crypto/fips140/Makefile create mode 100644 arch/x86/crypto/fips140/fips140-var-redirect.c
arch/x86/crypto/Makefile +22 -19
@@ -4,42 +4,42 @@ obj-$(CONFIG_CRYPTO_TWOFISH_586) += twofish-i586.o twofish-i586-y := twofish-i586-asm_32.o twofish_glue.o -obj-$(CONFIG_CRYPTO_TWOFISH_X86_64) += twofish-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_TWOFISH_X86_64) += twofish-x86_64.o twofish-x86_64-y := twofish-x86_64-asm_64.o twofish_glue.o -obj-$(CONFIG_CRYPTO_TWOFISH_X86_64_3WAY) += twofish-x86_64-3way.o +crypto-objs-$(CONFIG_CRYPTO_TWOFISH_X86_64_3WAY) += twofish-x86_64-3way.o twofish-x86_64-3way-y := twofish-x86_64-asm_64-3way.o twofish_glue_3way.o -obj-$(CONFIG_CRYPTO_TWOFISH_AVX_X86_64) += twofish-avx-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_TWOFISH_AVX_X86_64) += twofish-avx-x86_64.o twofish-avx-x86_64-y := twofish-avx-x86_64-asm_64.o twofish_avx_glue.o obj-$(CONFIG_CRYPTO_SERPENT_SSE2_586) += serpent-sse2-i586.o serpent-sse2-i586-y := serpent-sse2-i586-asm_32.o serpent_sse2_glue.o -obj-$(CONFIG_CRYPTO_SERPENT_SSE2_X86_64) += serpent-sse2-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_SERPENT_SSE2_X86_64) += serpent-sse2-x86_64.o serpent-sse2-x86_64-y := serpent-sse2-x86_64-asm_64.o serpent_sse2_glue.o -obj-$(CONFIG_CRYPTO_SERPENT_AVX_X86_64) += serpent-avx-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_SERPENT_AVX_X86_64) += serpent-avx-x86_64.o serpent-avx-x86_64-y := serpent-avx-x86_64-asm_64.o serpent_avx_glue.o -obj-$(CONFIG_CRYPTO_SERPENT_AVX2_X86_64) += serpent-avx2.o +crypto-objs-$(CONFIG_CRYPTO_SERPENT_AVX2_X86_64) += serpent-avx2.o serpent-avx2-y := serpent-avx2-asm_64.o serpent_avx2_glue.o -obj-$(CONFIG_CRYPTO_CAMELLIA_X86_64) += camellia-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_CAMELLIA_X86_64) += camellia-x86_64.o camellia-x86_64-y := camellia-x86_64-asm_64.o camellia_glue.o -obj-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX_X86_64) += camellia-aesni-avx-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX_X86_64) += camellia-aesni-avx-x86_64.o camellia-aesni-avx-x86_64-y := camellia-aesni-avx-asm_64.o camellia_aesni_avx_glue.o -obj-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX2_X86_64) += camellia-aesni-avx2.o +crypto-objs-$(CONFIG_CRYPTO_CAMELLIA_AESNI_AVX2_X86_64) += camellia-aesni-avx2.o camellia-aesni-avx2-y := camellia-aesni-avx2-asm_64.o camellia_aesni_avx2_glue.o -obj-$(CONFIG_CRYPTO_BLOWFISH_X86_64) += blowfish-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_BLOWFISH_X86_64) += blowfish-x86_64.o blowfish-x86_64-y := blowfish-x86_64-asm_64.o blowfish_glue.o -obj-$(CONFIG_CRYPTO_CAST5_AVX_X86_64) += cast5-avx-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_CAST5_AVX_X86_64) += cast5-avx-x86_64.o cast5-avx-x86_64-y := cast5-avx-x86_64-asm_64.o cast5_avx_glue.o -obj-$(CONFIG_CRYPTO_CAST6_AVX_X86_64) += cast6-avx-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_CAST6_AVX_X86_64) += cast6-avx-x86_64.o cast6-avx-x86_64-y := cast6-avx-x86_64-asm_64.o cast6_avx_glue.o -obj-$(CONFIG_CRYPTO_AEGIS128_AESNI_SSE2) += aegis128-aesni.o +crypto-objs-$(CONFIG_CRYPTO_AEGIS128_AESNI_SSE2) += aegis128-aesni.o aegis128-aesni-y := aegis128-aesni-asm.o aegis128-aesni-glue.o -obj-$(CONFIG_CRYPTO_AES_NI_INTEL) += aesni-intel.o +crypto-objs-$(CONFIG_CRYPTO_AES_NI_INTEL) += aesni-intel.o aesni-intel-y := aesni-intel_asm.o aesni-intel_glue.o aesni-intel-$(CONFIG_64BIT) += aes-ctr-avx-x86_64.o \ aes-gcm-aesni-x86_64.o \
@@ -47,17 +47,20 @@ aesni-intel-$(CONFIG_64BIT) += aes-ctr-avx-x86_64.o \ aes-gcm-vaes-avx512.o \ aes-xts-avx-x86_64.o -obj-$(CONFIG_CRYPTO_SM4_AESNI_AVX_X86_64) += sm4-aesni-avx-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_SM4_AESNI_AVX_X86_64) += sm4-aesni-avx-x86_64.o sm4-aesni-avx-x86_64-y := sm4-aesni-avx-asm_64.o sm4_aesni_avx_glue.o -obj-$(CONFIG_CRYPTO_SM4_AESNI_AVX2_X86_64) += sm4-aesni-avx2-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_SM4_AESNI_AVX2_X86_64) += sm4-aesni-avx2-x86_64.o sm4-aesni-avx2-x86_64-y := sm4-aesni-avx2-asm_64.o sm4_aesni_avx2_glue.o -obj-$(CONFIG_CRYPTO_ARIA_AESNI_AVX_X86_64) += aria-aesni-avx-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_ARIA_AESNI_AVX_X86_64) += aria-aesni-avx-x86_64.o aria-aesni-avx-x86_64-y := aria-aesni-avx-asm_64.o aria_aesni_avx_glue.o -obj-$(CONFIG_CRYPTO_ARIA_AESNI_AVX2_X86_64) += aria-aesni-avx2-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_ARIA_AESNI_AVX2_X86_64) += aria-aesni-avx2-x86_64.o aria-aesni-avx2-x86_64-y := aria-aesni-avx2-asm_64.o aria_aesni_avx2_glue.o -obj-$(CONFIG_CRYPTO_ARIA_GFNI_AVX512_X86_64) += aria-gfni-avx512-x86_64.o +crypto-objs-$(CONFIG_CRYPTO_ARIA_GFNI_AVX512_X86_64) += aria-gfni-avx512-x86_64.o aria-gfni-avx512-x86_64-y := aria-gfni-avx512-asm_64.o aria_gfni_avx512_glue.o + +# FIPS 140 kernel module +obj-$(CONFIG_CRYPTO_FIPS140_EXTMOD) += fips140/
arch/x86/crypto/fips140/Makefile new file +14 -0
@@ -0,0 +1,14 @@ + +crypto-objs-y += fips140-var-redirect-fips.o + +obj-y += fips140-var-redirect-main.o + +# Explicit rules to compile same source to different objects +$(obj)/fips140-var-redirect-main.o: $(src)/fips140-var-redirect.c FORCE + $(call if_changed_rule,cc_o_c) + +$(obj)/fips140-var-redirect-fips.o: $(src)/fips140-var-redirect.c FORCE + $(call if_changed_rule,cc_o_c) + +CFLAGS_fips140-var-redirect-main.o += -I$(srctree) +CFLAGS_fips140-var-redirect-fips.o += -I$(srctree)
arch/x86/crypto/fips140/fips140-var-redirect.c new file +0 -0
Signature · 1 line
2.47.3
PATCH v2 18/19 arm64/crypto: convert exported symbols in arm64 crypto to pluggable symbols
In reply to Jay WangInclude arm64 architecture-specific crypto source files in fips140.ko by converting their Makefile rules from obj-y to crypto-objs-y, and apply the pluggable interface introduced in the earlier patch so that symbols defined in fips140.ko can still be referenced by vmlinux.
For exported function symbols, the --wrap linker mechanism automatically redirects all references in vmlinux to trampolines, so no source tree modifications are needed beyond ensuring each function has an EXPORT_SYMBOL.
For exported variable symbols, an architecture-specific fips140-var-redirect.c is introduced under arch/arm64/crypto/fips140/ to hold DEFINE_CRYPTO_VAR_STUB() definitions. This file is compiled twice: once for vmlinux (as the "outlet" providing the placeholder pointers) and once for fips140.ko (as the "plug" populating them with real addresses via the __crypto_var_keys section).
--- arch/arm64/crypto/Makefile | 23 +++++++++++-------- arch/arm64/crypto/fips140/Makefile | 14 +++++++++++ .../crypto/fips140/fips140-var-redirect.c | 0 3 files changed, 27 insertions(+), 10 deletions(-) create mode 100644 arch/arm64/crypto/fips140/Makefile create mode 100644 arch/arm64/crypto/fips140/fips140-var-redirect.c
arch/arm64/crypto/Makefile +13 -10
@@ -5,32 +5,35 @@ # Copyright (C) 2014 Linaro Ltd <[email protected]> # -obj-$(CONFIG_CRYPTO_SM4_ARM64_CE) += sm4-ce-cipher.o +crypto-objs-$(CONFIG_CRYPTO_SM4_ARM64_CE) += sm4-ce-cipher.o sm4-ce-cipher-y := sm4-ce-cipher-glue.o sm4-ce-cipher-core.o -obj-$(CONFIG_CRYPTO_SM4_ARM64_CE_BLK) += sm4-ce.o +crypto-objs-$(CONFIG_CRYPTO_SM4_ARM64_CE_BLK) += sm4-ce.o sm4-ce-y := sm4-ce-glue.o sm4-ce-core.o -obj-$(CONFIG_CRYPTO_SM4_ARM64_CE_CCM) += sm4-ce-ccm.o +crypto-objs-$(CONFIG_CRYPTO_SM4_ARM64_CE_CCM) += sm4-ce-ccm.o sm4-ce-ccm-y := sm4-ce-ccm-glue.o sm4-ce-ccm-core.o -obj-$(CONFIG_CRYPTO_SM4_ARM64_CE_GCM) += sm4-ce-gcm.o +crypto-objs-$(CONFIG_CRYPTO_SM4_ARM64_CE_GCM) += sm4-ce-gcm.o sm4-ce-gcm-y := sm4-ce-gcm-glue.o sm4-ce-gcm-core.o -obj-$(CONFIG_CRYPTO_SM4_ARM64_NEON_BLK) += sm4-neon.o +crypto-objs-$(CONFIG_CRYPTO_SM4_ARM64_NEON_BLK) += sm4-neon.o sm4-neon-y := sm4-neon-glue.o sm4-neon-core.o -obj-$(CONFIG_CRYPTO_GHASH_ARM64_CE) += ghash-ce.o +crypto-objs-$(CONFIG_CRYPTO_GHASH_ARM64_CE) += ghash-ce.o ghash-ce-y := ghash-ce-glue.o ghash-ce-core.o -obj-$(CONFIG_CRYPTO_AES_ARM64_CE_CCM) += aes-ce-ccm.o +crypto-objs-$(CONFIG_CRYPTO_AES_ARM64_CE_CCM) += aes-ce-ccm.o aes-ce-ccm-y := aes-ce-ccm-glue.o aes-ce-ccm-core.o -obj-$(CONFIG_CRYPTO_AES_ARM64_CE_BLK) += aes-ce-blk.o +crypto-objs-$(CONFIG_CRYPTO_AES_ARM64_CE_BLK) += aes-ce-blk.o aes-ce-blk-y := aes-glue-ce.o -obj-$(CONFIG_CRYPTO_AES_ARM64_NEON_BLK) += aes-neon-blk.o +crypto-objs-$(CONFIG_CRYPTO_AES_ARM64_NEON_BLK) += aes-neon-blk.o aes-neon-blk-y := aes-glue-neon.o -obj-$(CONFIG_CRYPTO_AES_ARM64_BS) += aes-neon-bs.o +crypto-objs-$(CONFIG_CRYPTO_AES_ARM64_BS) += aes-neon-bs.o aes-neon-bs-y := aes-neonbs-core.o aes-neonbs-glue.o + +# FIPS 140 kernel module +obj-$(CONFIG_CRYPTO_FIPS140_EXTMOD) += fips140/
arch/arm64/crypto/fips140/Makefile new file +14 -0
@@ -0,0 +1,14 @@ + +crypto-objs-y += fips140-var-redirect-fips.o + +obj-y += fips140-var-redirect-main.o + +# Explicit rules to compile same source to different objects +$(obj)/fips140-var-redirect-main.o: $(src)/fips140-var-redirect.c FORCE + $(call if_changed_rule,cc_o_c) + +$(obj)/fips140-var-redirect-fips.o: $(src)/fips140-var-redirect.c FORCE + $(call if_changed_rule,cc_o_c) + +CFLAGS_fips140-var-redirect-main.o += -I$(srctree) +CFLAGS_fips140-var-redirect-fips.o += -I$(srctree)
arch/arm64/crypto/fips140/fips140-var-redirect.c new file +0 -0
Signature · 1 line
2.47.3
PATCH v2 19/19 Add standalone crypto kernel module technical documentation
In reply to Jay WangTechnical guide covering implementation details and usage of the standalone crypto kernel module feature.
--- crypto/fips140/README | 404 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 404 insertions(+) create mode 100644 crypto/fips140/README
crypto/fips140/README new file +404 -0
@@ -0,0 +1,404 @@ +## 1. Introduction + +Amazon Linux is releasing a new kernel feature that converts the previously built-in kernel crypto subsystem into a standalone kernel module. This module becomes the carrier of the kernel crypto subsystem and can be loaded at early boot to provide the same functionality as the original built-in crypto. The primary motivation for this modularization is to streamline Federal Information Processing Standards (FIPS) validation, a critical cryptographic certification for cloud computing users doing business with the U.S. government. + +In a bit more detail, previously, FIPS certification was tied to the entire kernel image, meaning non-crypto updates could potentially invalidate certification. With this feature, FIPS certification is tied only to the crypto module. Therefore, once the module is certified, loading this certified module on newer kernels automatically makes those kernels FIPS-certified. As a result, this approach can save re-certification costs and 12-18 months of waiting time by reducing the need for repeated FIPS re-certification cycles. + +This document provides technical details on how this feature is designed and implemented for users or developers who are interested in developing upon it, and is organized as follows: +- Section 2 - Getting Started: Quick start on how to enable the feature +- Section 3 - Workflow Overview: Changes this feature brings to build and runtime +- Section 4 - Design Implementation Details: Technical deep-dive into each component +- Section 5 - Customizing and Extending Crypto Module: How to select crypto to be included and extend to new crypto/architectures +- Section 6 - Reusing a Certified Module in Practice: Different reuse and maintenance strategies and their tradeoffs + +## 2. Getting Started + +This section provides a quick start guide for developers on how to enable, compile and use the standalone cryptography module feature. + +### 2.1 Basic Configuration + +The feature is controlled by a single configuration option: +``` +CONFIG_CRYPTO_FIPS140_EXTMOD=y +``` +What it does: When enabled, automatically redirects a set of cryptographic algorithms from the main kernel into a standalone module `crypto/fips140/fips140.ko`. The cryptographic algorithms that are redirected need to satisfy all the following conditions, otherwise the cryptography will remain in its original form: +1. Must be configured as built-in (i.e., `CONFIG_CRYPTO_*=y`). This means cryptography already configured as modular (i.e., `CONFIG_CRYPTO_*=m`) are not redirected as they are already modularized. +2. Must be among a list, which can be customized by developers as described in Section 5. + +When disabled, the kernel behaves as before. + +### 2.2 Build Process + +Once `CONFIG_CRYPTO_FIPS140_EXTMOD=y` is set, no additional steps are required. The standalone module will be built automatically as part of the standard kernel build process: +``` +make -j$(nproc) +# or +make vmlinux +``` +**What happens automatically (No user action required):** +1. Build the module as `crypto/fips140/fips140.ko` +2. The cryptography module will be loaded at boot time +3. All kernel cryptographic services will provide the same functionality as before (i.e., prior to introducing this new feature) once boot completes. + +### 2.3 Advanced Configuration Options + +**Using External Cryptography Module:** +``` +CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE=y +``` +By default, `CONFIG_CRYPTO_FIPS140_EXTMOD_SOURCE` is not set, meaning the freshly built cryptography module is used. Otherwise, the pre-built standalone cryptography module from `fips140_build/crypto/fips140/fips140.ko` and modular cryptography such as `fips140_build/crypto/aes.ko` (need to manually place pre-built modules in these locations before the build) are included in kernel packaging (e.g., during `make modules_install`) and are used at later boot time. + +**Dual Version Support:** +``` +CONFIG_CRYPTO_FIPS140_DUAL_VERSION=y +``` +Encapsulate two versions of `fips140.ko` into kernel: one is freshly built for non-FIPS mode usage, another is pre-built specified by `fips140_build/crypto/fips140/fips140.ko` for FIPS mode usage. The appropriate version is selected and loaded at boot time based on boot time FIPS mode status. + +### 2.4 Verification + +To verify the feature is working, after install and boot with the new kernel: +``` +# Check if fips140.ko module is loaded +lsmod | grep fips140 +# Check if crypto algorithms are served by the fips140 module +cat /proc/crypto | grep module | grep fips140 +``` + +## 3. Workflow Overview + +This section provides an overview without delving into deep technical details of the changes the standalone cryptography module feature introduces. When this feature is enabled, it introduces changes to both the kernel build and booting process. + +3.1 Build-Time Changes + +Kernel cryptography subsystem consists of both cryptography management infrastructure (e.g., `crypto/api.c`, `crypto/algapi.c`, etc), along with hundreds of different cryptography algorithms (e.g., `crypto/arc4.c`). + +**Traditional Build Process:** +Traditionally, cryptography management infrastructure are always built-in to the kernel, while cryptographic algorithms can be configured to be built either as built-in (`CONFIG_CRYPTO_*=y`) or as separate modular (`CONFIG_CRYPTO_*=m`) `.ko` file depending on kernel configuration: +As a result, the builtin cryptography management infrastructure and cryptographic algorithms are statically linked into the kernel binary: +``` +cryptographic algorithms source files → compiled as .o objfiles → linked into vmlinux → single kernel binary +``` +**With Standalone Cryptography Module:** +This feature automatically transforms the builtin cryptographic components into a standalone cryptography module, `fips140.ko`. To do so, it develops a new kernel build rule `crypto-objs-$(CONFIG_CRYPTO_*)` such that, once this build rule is applied to a cryptographic algorithm, such cryptographic algorithm will be automatically collected into the cryptography module if it is configured as built-in (i.e, `CONFIG_CRYPTO_*=y`), for example: +``` +// in crypto/asymmetric_keys/Makefile +- obj-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o ++ crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o +``` +Such build change allows the modularization transformation to only affect selected cryptographic algorithms (i.e, where the `crypto-objs-$(CONFIG_CRYPTO_*`) is applied). + +Then, after the `fips140.ko` is generated, it will be embedded back into main kernel vmlinux as a replacement part. The purpose of this embedding, instead of traditionally putting the `fips140.ko` into filesystem, is a preparation to allow the module to be loaded early enough even before the filesystem is ready. + +The new build process is illustrated below. +``` +cryptographic algorithms source files → compiled as .o objfiles → automatically collected and linked into fips140.ko → embedded fips140.ko into vmlinux as a replaceable binary +``` + +### 3.2 Runtime Changes + +**Traditional Boot Process:** +The kernel initializes the cryptographic subsystem early during boot, executing each cryptographic initialization routine accordingly. These initialization routines may depend on other cryptographic components or other kernel subsystems, so their invocation follows a well-defined execution order to ensure they are initialized before their first use. +``` +kernel starts → cryptography subsystem initialization → cryptography subsystem available → other components use cryptography +``` +**With Standalone Cryptography Module:** +At the start of kernel boot, compared to a regular kernel, the first major change introduced by this feature is that no cryptography services are initially available — since the entire cryptography subsystem has been decoupled from the main kernel. +To ensure that the cryptography subsystem becomes available early enough (before the first kernel component that requires cryptography services), the standalone cryptography kernel module must be loaded at a very early stage, even before the filesystem becomes available. + +However, the regular module loading mechanism relies on placing kernel modules in the filesystem and loading them from there, which creates a chicken-and-egg problem — the cryptography module cannot be loaded until the filesystem is ready, yet some kernel components may require cryptography services even before that point. + +To address this, the second change introduced by this feature is that the cryptography kernel module is loaded directly from memory, leveraging the earlier compilation changes that embed the module binary into the main kernel image. Afterward, the feature includes a “plug-in” mechanism that connects the decoupled cryptography subsystem back to the main kernel, ensuring that kernel cryptography users can correctly locate and invoke the cryptography routine entry points. + +Finally, to ensure proper initialization, the feature guarantees that all cryptography algorithms and the cryptography management infra execute their initialization routines in the exact same order as they would if they were built-in. + +The process described above is illustrated below. +``` +kernel starts → no cryptography available → load fips140.ko from memory → plug cryptography back to kernel → module initialization → cryptographic services available → other components use cryptography +``` + +## 4. Design Implementation Details + +While the earlier sections provide a holistic view of how this feature shapes the kernel, this section provides deeper design details on how these functionalities are realized. There are three key design components: +1. A specialized compile rule that automatically compiles and collects all built-in cryptographic algorithm object files to generate the final module binary under arbitrary kernel configurations, and then embeds the generated binary into the main kernel image for early loading. +2. A mechanism to convert interactions between the cryptography subsystem and the main kernel into a pluggable interface. +3. A module loading and initialization process that ensures the cryptography subsystem is properly initialized as if it were built-in. + +### 4.1. Specialized Compilation System + +**Automatic Collection and Linking of Built-in Cryptographic Algorithm Objects:** +The first step in generating the `fips140.ko` module is to compile and collect built-in cryptographic components (i.e., those specified by `CONFIG_CRYPTO_*=y`). +Traditionally, the existing module build process requires all module components (e.g., source files) to reside in a single directory. However, this approach is not suitable for our case, where hundreds of cryptographic algorithm source files are scattered across multiple directories. + +A naïve approach would be to create a separate Makefile that duplicates the original build rules with adjusted paths. +However, this method is not scalable due to the large number of cryptographic build rules, many of which are highly customized and can vary under different Kconfig settings, making such a separate Makefile even more complex. +Moreover, this approach cannot ensure that built-in cryptographic algorithms are completely removed from the main kernel, which would result in redundant cryptographic code being included in both the kernel and the module. + +To tackle this challenge, we automated the object collection and linking process by introducing special build logic for the kernel cryptography subsystem. +Specifically, to automatically collect cryptography object files while preserving their original compilation settings (such as flags, headers, and paths), we introduced a new compilation rule: +``` +crypto-objs-y += *.o +``` +This replaces the original `obj-y += *.o` rule in cryptography Makefiles later, for example: +``` +// in crypto/asymmetric_keys/Makefile +- obj-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o ++ crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o +asymmetric_keys-y := \ + asymmetric_type.o \ + restrict.o \ + signature.o +``` +in the cryptography subsystem Makefiles, allowing most of the existing Makefile logic to be reused. +As a result, when the standalone cryptography module feature is enabled, any cryptographic algorithm configured as built-in (for example, `crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o` where `CONFIG_ASYMMETRIC_KEY_TYPE=y`) will be automatically collected and linked into a single final object binary, `fips140.o`. +During this process, a special compilation flag (`-DFIPS_MODULE=1`) is applied to instruct each object file to be compiled in a module-specific manner. This flag will later be used to generate the pluggable interface on both the main kernel side and the module side from the same source code. + +The implementation details are as follows: it follows a similar methodology used by the `obj-y` collection process for building `vmlinux.o`. The `crypto-objs-y` rule is placed in `scripts/Makefile.build`, which is executed by each directory Makefile to collect the corresponding crypto object files. Each directory then creates a `crypto-module.a` archive that contains all `crypto-objs-y += <object>.o` files under that directory. In the parent directories, these `crypto-module.a` archives are recursively included into the parent’s own `crypto-module.a`, and this process continues upward until the final `fips140.o` is generated. + +**A Separate Module Generation Pipeline for Building the Final Kernel Module from Linked Cryptographic Algorithm Object:** +With the linked cryptographic algorithm object (i.e., `fips140.o`), the next step is to generate the final kernel module, `fips140.ko`. + +A direct approach would be to inject the `fips140.ko` module build into the existing modules generation pipeline (i.e., `make modules`) by providing our pre-generated `fips140.o`. However, we choose not to do this because it would create a circular make rule dependency (which is invalid in Makefiles and causes build failures), resulting in mutual dependencies between the modules and vmlinux targets (i.e., `modules:vmlinux` and `vmlinux:modules` at the same time). +This happens for the following reasons: +1. Since we will later embed `fips140.ko` into the final kernel image (as described in the next section), we must make vmlinux depend on `fips140.ko`. In other words: `vmlinux: fips140.ko`. +2. When the kernel is built with `CONFIG_DEBUG_INFO_BTF_MODULES=y`, it requires: modules: vmlinux. This is because `CONFIG_DEBUG_INFO_BTF_MODULES=y` takes vmlinux as input to generate BTF info for the module, and inserts such info into the `.ko` module by default. +3. If we choose to inject `fips140.ko` into make modules, this would create a make rule dependency: `fips140.ko: modules`. Combined with items 1 and 2, this eventually creates an invalid circular dependency between vmlinux and modules. + +Due to these reasons, the design choice is to use a separate make pipeline (defined as `fips140-ready` in the Makefile). This new pipeline reuses the same module generation scripts used by make modules but adds additional logic in `scripts/Makefile.{modfinal|modinst|modpost}` and `scripts/mod/modpost.c` to handle module symbol generation and verification correctly. + +**A Seamless Process That Embeds the Generated Binary Into the Main Kernel Image for Early Loading:** +As mentioned earlier, in order to load the standalone cryptography module early in the boot process—before the filesystem is ready—the module binary must be embedded into the final kernel image (i.e., vmlinux) so that it can be loaded directly from memory. +We intend for this embedding process to be completely seamless and automatically triggered whenever vmlinux is built (i.e., during `make vmlinux`). + +To achieve this, the feature adds a Make dependency rule so that vmlinux depends on `fips140.ko`. +It also modifies the vmlinux link rules (i.e., `arch/<arch>/kernel/vmlinux.lds.S`, `scripts/Makefile.vmlinux`, and `scripts/link-vmlinux.sh`) so that the generated module binary is finally combined with `vmlinux.o`. + +In addition, we allow multiple cryptography module binary versions (for example, a certified cryptography binary and a latest, up-to-date but uncertified one) to be embedded into the main kernel image to serve different user needs. This design allows regular (non-FIPS) users to benefit from the latest cryptographic updates, while FIPS-mode users continue to use the certified cryptography module. + +To support this, we introduce an optional configuration, `CONFIG_CRYPTO_FIPS140_DUAL_VERSION`. When enabled, this option allows two cryptography module versions to be embedded within a single kernel build and ensures that the appropriate module is selected and loaded at boot time based on the system’s FIPS mode status. + +### 4.2. Pluggable Interface Between the Built-in Cryptography Subsystem and the Main Kernel + +Although the module binary (`fips140.ko`) has been embedded into the final kernel image (`vmlinux`) as described in the previous section, it is not linked to the kernel in any way. This is because `fips140.ko` is embedded in a data-only manner, so the main kernel cannot directly call any functions or access any data defined in the module binary. Since the main kernel and modules can only interact through exported symbols (i.e., via `EXPORT_SYMBOL()`), this also applies to the crypto kernel module — the main kernel can only interact with the crypto functions and variables defined in the crypto module through exported symbols, meaning these functions and variables must also have their symbols exported in the module after they are moved from the main kernel to the module. + +However, simply making these crypto symbols symbol-exported in the module without additional handling would cause the kernel to fail to compile. This is because the existing kernel module symbol resolution mechanism is essentially one-way: it supports symbols defined in the main kernel and referenced by kernel modules. However, it does not support the reverse case — symbols defined in a kernel module but used by the main kernel — which is exactly the crypto module case, as there are many crypto users still residing in the main kernel. The reason is that compilation of the main kernel requires all symbol addresses to be known to achieve a successful linking phase. + +To address this, we introduce a pluggable interface to support this reverse-direction symbol resolution between crypto symbols defined in the module and referenced by the main kernel, by placing **address placeholders** at all crypto usage points in the main kernel. These address placeholders are initially set to NULL during compilation to provide a concrete address that satisfies the linking phase. Then, during runtime, once the cryptography kernel module is loaded, these placeholders are updated to the correct addresses before their first use in the main kernel. In the rest of this section, we first introduce this pluggable interface mechanism, and then explain how to apply it to the built-in cryptographic algorithms and variables. + +**The Pluggable Interface Mechanism:** +There are two types of address holders used to achieve this pluggable interface: +- Function addresses (the majority): We use a trampoline to redirect the original jump instruction to another location whose target destination is held by the value of a function pointer. To avoid additional security concerns, such as the function pointer being arbitrarily modified, these function pointers are made `__ro_after_init` to ensure they cannot be modified after kernel init. We implement this function-address placeholder as the `DEFINE_CRYPTO_FN_REDIRECT()` wrapper. +- Variable addresses (the remaining smaller portion): For these, we use a pointer of the corresponding data type. We implement this address placeholder as the `DECLARE_CRYPTO_VAR()` and `DEFINE_CRYPTO_API_STUB()` wrappers: + +These wrappers are applied to each symbol-exported (i.e., `EXPORT_SYMBOL()`) cryptographic function and variable (details on how to apply them are described later). Once applied, the wrappers are compiled differently for the main kernel and for the built-in cryptographic algorithm source code—acting as the “outlet” and the “plug,” respectively—using different compilation flags (`-DFIPS_MODULE`) introduced by our customized build rules described earlier. + +As a result, the kernel can successfully compile even when the built-in cryptographic algorithms are removed, thanks to these address placeholders. At boot time, the placeholders initially hold NULL, but since no cryptography users exist at that stage, the kernel can still start booting correctly. After the cryptography module is loaded, the placeholders are dynamically updated to the correct addresses later (by `do_crypto_var()` and `do_crypto_fn()`, described in a later section). + +**Applying the Pluggable Interface Mechanism to Cryptographic Algorithms:** + +To apply these pluggable interface wrappers to a cryptographic algorithm and make them take effect, we follow the steps below (using `crypto/asymmetric_keys/asymmetric_type.c` as an example): +1. **Apply `crypto-objs-y` compile rule to the cryptographic algorithm:** +``` +// in crypto/asymmetric_keys/Makefile +- obj-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o ++ crypto-objs-$(CONFIG_ASYMMETRIC_KEY_TYPE) += asymmetric_keys.o +asymmetric_keys-y := \ + asymmetric_type.o \ + restrict.o \ + signature.o +``` +2. **Locate the communication point between the cryptographic algorithm and the main kernel:** + +The cryptography subsystem is designed such that most interactions between the main kernel and cryptographic algorithms occur through exported symbols using `EXPORT_SYMBOL()` wrappers. +This kernel design exists because most cryptographic algorithm implementations must support both built-in and modular modes. + +Consequently, the cryptographic functions and variables exported by `EXPORT_SYMBOL()` are a well-defined and identifiable interface between the cryptography subsystem and the main kernel: +``` +// in crypto/asymmetric_keys/asymmetric_type.c +//Exported cryptographic function: +bool asymmetric_key_id_same(const struct asymmetric_key_id *kid1, + const struct asymmetric_key_id *kid2) {...} +EXPORT_SYMBOL_GPL(asymmetric_key_id_same); +//Exported cryptographic variable: +struct key_type key_type_asymmetric = {...}; +EXPORT_SYMBOL_GPL(key_type_asymmetric); +``` + +3. **Redirect crypto symbol references in the main kernel to address placeholders:** + +With the placeholders in place, the remaining problem is directing the main kernel to use them rather than the original symbols. Since all crypto users must include the corresponding header files to obtain function and variable declarations, the headers are a natural place to perform this redirection. Each declaration is transformed using a macro that hooks it to the corresponding placeholder. + +For exported variable symbols (a small number, ~10 symbols), their declaration in the header file is replaced with the `DECLARE_CRYPTO_VAR()` wrapper to redirect variable access from a concrete address to a placeholder: +``` +// in include/keys/asymmetric-type.h +// for exported cryptographic variables: +- struct key_type key_type_asymmetric; ++ DECLARE_CRYPTO_VAR(CONFIG_ASYMMETRIC_KEY_TYPE, key_type_asymmetric, struct key_type, ); ++ #if defined(CONFIG_CRYPTO_FIPS140_EXTMOD) && !defined(FIPS_MODULE) && IS_BUILTIN(CONFIG_ASYMMETRIC_KEY_TYPE) ++ #define key_type_asymmetric (*((struct key_type*)CRYPTO_VAR_NAME(key_type_asymmetric))) ++ #endif +``` +By doing so, we can automatically force all cryptography users to go through the placeholders, because those users already include the same header file. +The wrapper also takes the cryptographic algorithm Kconfig symbol as a parameter, so that when a cryptographic algorithm is built as a module (for example, `CONFIG_ASYMMETRIC_KEY_TYPE=m`), the original function declarations remain unchanged and are not affected. + +For exported function symbols (the majority, ~hundreds), a similar approach could be taken, but instead we use an automated method to redirect function address usage to placeholders during the kernel compilation process. This makes the crypto module implementation less intrusive to the kernel source tree, as no header file modifications are needed. To achieve this, a linker option `--wrap=<symbols-to-redirect>` is leveraged to rename all uses of crypto functions in the main kernel to dedicated trampolines that act as address placeholders. As a consequence, all references to crypto function symbols are automatically redirected to the address placeholders, avoiding mass intrusive changes to the mainline kernel source tree. + +4. **Add the address-placeholder definition wrappers into a dedicated file `fips140-var-redirect.c`:** + +After redirecting crypto users to use address placeholders, we also need to add the definitions of those address placeholders. + +For exported variable symbols (a small number, ~10 symbols), add the placeholder definition wrapper `DEFINE_CRYPTO_VAR_STUB` to a dedicated file `fips140-var-redirect.c`. +``` +// in crypto/fips140/fips140-var-redirect.c +// for exported cryptographic variables: ++ #undef key_type_asymmetric ++ DEFINE_CRYPTO_VAR_STUB(key_type_asymmetric); ++ #endif +``` +This file will be compiled separately and acts as both the “outlet” and the “plug” for the main kernel and the cryptography module, respectively. + +For exported function symbols (the majority, ~hundreds), a similar wrapper `DEFINE_CRYPTO_FN_REDIRECT()` is used, but again, the application of this wrapper is automated, so there is no need to manually apply it. Instead, it is generated automatically by the script `crypto/fips140/gen-fips140-fn-redirect.sh` on every kernel build. + +We apply the above steps to both architecture-independent and architecture-specific cryptographic algorithms. + +### 4.3. Initialization Synchronization + +To ensure the embedded `fips140.ko` module binary provides the same cryptography functionality as the regular kernel, the kernel needs: +1. A module loader to load the module binary directly from memory, +2. A mechanism to plug the module back into the kernel by updating the address placeholders, and +3. Correct cryptography subsystem initialization, as if the cryptographic algorithms were still built-in. + +**Directly Load Module Binary from Memory:** +Regular modules are loaded from the filesystem and undergo signature verification on the module binary, which relies on cryptographic operations. However, since we have already fully decoupled the cryptography subsystem, we must skip this step for this `fips140.ko` module. +To achieve this, we add a new loader function `load_crypto_module_mem()` that can load the module binary directly from memory at the designed address without checking the signature. Since the module binary is embedded into main kernel in an ELF section, as specified in the linker script: +``` +// in arch/<arch>/kernel/vmlinux.lds.S + .fips140_embedded : AT(ADDR(.fips140_embedded) - LOAD_OFFSET) { + . = ALIGN(8); + _binary_fips140_ko_start = .; + KEEP(*(.fips140_module_data)) + _binary_fips140_ko_end = .; + } +``` +Therefore, the runtime memory address of the module can be accessed directly by the module loader to invoke the new loader function `load_crypto_module_mem()`. + +**Plug Back the Module by Updating Address Placeholder Values:** +To update the address placeholders in the main kernel to the correct addresses matching the loaded module, after compilation the placeholder values are placed into dedicated key-value data structures, which reside in ELF sections `__crypto_fn_keys` and `__crypto_var_keys`. +This can be seen from the definition of the placeholder's key-value data structure: +``` +#define __CRYPTO_FN_KEY(sym) \ + extern void *__fips140_fn_ptr_##sym; \ + static struct _crypto_fn_key __##sym##_fn_key \ + __used \ + __section("__crypto_fn_keys") \ // Place in a dedicated ELF Section + __aligned(__alignof__(struct _crypto_fn_key)) = { \ + .ptr = (void **)&__fips140_fn_ptr_##sym, \ + .func = (void *)&sym, \ + }; + +#define DEFINE_CRYPTO_VAR_STUB(name) \ + static struct crypto_var_key __crypto_##name##_var_key \ + __used \ + __section("__crypto_var_keys") \ // Place in a dedicated ELF Section + __aligned(__alignof__(struct crypto_var_key)) = \ + { \ + .ptr = &CRYPTO_VAR_NAME(name), \ + .var = (void*)&name, \ + }; +``` +The purpose of doing this is to allow the main kernel to quickly locate the placeholders and update them to the correct addresses. The update functions are defined as `do_crypto_var()` and `do_crypto_fn()`, which are executed at module load. + +As a result, all cryptography users in the main kernel can now call the cryptographic functions as if they were built-in. + +**Initialize Cryptography Subsystem as if it Were Built-in:** +Cryptographic components must be properly initialized before use, and this initialization is typically achieved through dedicated initialization functions (e.g., `module_init(crypto_init_func)` or `late_initcall(crypto_init_func)`). Traditionally, these init functions are executed automatically as part of the kernel boot phase. However, now that they are moved to a crypto module, there needs to be a way to collect and execute them. + +To collect these init functions, the init wrappers (e.g., `module_init()` and `late_initcall`) are modified to automatically place the wrapped crypto init function into a dedicated list in the crypto module, for example: +``` +// in include/linux/module.h +#define subsys_initcall(fn) \ + static initcall_t __used __section(".fips_initcall0") \ // a dedicated list + __fips_##fn = fn; + +#define module_init(initfn) \ + static initcall_t __used __section(".fips_initcall1") \ // a dedicated list + __fips_##initfn = initfn; +``` +By doing so, all init functions are now aggregated for execution by `run_initcalls()` at module load. + +Besides collecting these crypto init functions, rather than simply executing them, another key consideration is that their execution order often has strict requirements. In other words, for these collected crypto init functions, we must ensure that their initialization order is preserved as before because failure to follow the correct order can result in kernel panic. + +To address this, we introduce a synchronization mechanism between the main kernel and the module to ensure all cryptographic algorithms are executed in the correct kernel boot phase. In more details, we spawn the module initialization process `fips_loader_init()` as an async thread `fips140_sync_thread()`, in which we call `run_initcalls()` to execute the initialization calls of each cryptographic algorithm. +Then, we introduce synchronization helpers such as `wait_until_fips140_level_sync(int level)` to ensure the initialization order of all cryptographic algorithms is synchronized with the main kernel. + +## 5. Customization and Extension of Cryptography Module + +This section describes how developers can customize which cryptographic algorithms are included in the standalone cryptography module, as well as extend this feature to other cryptographic algorithms or hardware architectures. + +### 5.1. Cryptography Selection Mechanism + +The feature automatically includes cryptographic algorithms that meet specific criteria: +1. **Built-in Configuration**: Only cryptographic algorithms configured as `CONFIG_CRYPTO_*=y` are candidates for inclusion +2. **Explicit Inclusion**: Cryptographic algorithms must be explicitly converted using the `crypto-objs-$(CONFIG__CRYPTO_*`) build rule + +### 5.2. Extend Support to New Cryptographic Algorithms + +To extend support to a new cryptographic algorithm in the standalone module, follow these steps: + +**Step 1: Update the Makefile** +``` +# in crypto/[algorithm]/Makefile +- obj-$(CONFIG_CRYPTO_ALGORITHM) += algorithm.o ++ crypto-objs-$(CONFIG_CRYPTO_ALGORITHM) += algorithm.o +``` +For Architecture-Specific Cryptographic Algorithms: +- Apply the `crypto-objs-` rule in the appropriate `arch/*/crypto/Makefile` + +**Step 2: Add Pluggable Interface Support** +If the cryptographic algorithm has symbol-exported variables via `EXPORT_SYMBOL()`, add the pluggable interface wrappers. There is no need to manually apply wrappers for symbol-exported functions: +``` +// Example: in include/keys/asymmetric-type.h +// for exported cryptographic variables: +- struct key_type key_type_asymmetric; ++ DECLARE_CRYPTO_VAR(CONFIG_ASYMMETRIC_KEY_TYPE, key_type_asymmetric, struct key_type, ); ++ #if defined(CONFIG_CRYPTO_FIPS140_EXTMOD) && !defined(FIPS_MODULE) && IS_BUILTIN(CONFIG_ASYMMETRIC_KEY_TYPE) ++ #define key_type_asymmetric (*((struct key_type*)CRYPTO_VAR_NAME(key_type_asymmetric))) ++ #endif +``` +Then, add the corresponding stubs in `crypto/fips140/fips140-var-redirect.c`: +``` ++ #undef key_type_asymmetric ++ DEFINE_CRYPTO_VAR_STUB(key_type_asymmetric); ++ #endif +``` +For Architecture-Specific Cryptographic Algorithms: +- Include architecture-specific stubs in `arch/*/crypto/fips140/fips140-var-redirect.c`: + +### 5.3. Architecture-Specific Extensions + +**Extending to New Architectures:** +Currently supported architectures are x86_64 and ARM64. To extend this feature to additional architectures: +1. **Update Linker Scripts**: Add ELF sections in `arch/[new-arch]/kernel/vmlinux.lds.S`: +``` +.fips140_embedded : AT(ADDR(.fips140_embedded) - LOAD_OFFSET) { + . = ALIGN(8); + _binary_fips140_ko_start = .; + KEEP(*(.fips140_module_data)) + _binary_fips140_ko_end = .; +} +``` +2. **Create Architecture-Specific Files**: Set up `arch/[new-arch]/crypto/fips140/` directory with Makefile and `fips140-var-redirect.c` following the pattern used in x86_64 and ARM64. + +## 6. Reusing a certified module in practice + +With the crypto subsystem restructured as a loadable module, a previously certified module can be reused across kernel updates. Crypto development does not freeze in the meantime — updated modules can always be submitted for fresh certification. The reuse of an already-certified module is intended to simply bridge the gap until the new certification arrives, that is, using the old one before the new one gets certified. How much of the certified module to reuse, however, involves a tradeoff between crypto feature availability, certification turnaround time, and engineering effort. + +The most conservative option is no reuse at all: abandon the previous certification and submit the updated crypto module for a full FIPS certification. This allows the crypto subsystem to benefit from the latest upstream changes, but at the cost of the full 12-to-18-month waiting period, according to experiences from downstream distributions. + +At the opposite end, distributions can choose to reuse the certified module binary entirely on a newer kernel. This enables the new kernel to be validated on day one. The tradeoff is clear: besides forgoing crypto updates, engineers must ensure ABI compatibility between the updated kernel and the module. + +A middle ground is to reuse only the source code of the certified module, freezing it and recompiling against the updated main kernel. Since the source code remains unchanged, the new module can go through a Non-Security Relevant ([NSRL](https://csrc.nist.gov/csrc/media/Projects/cryptographic-module-validation-program/documents/fips%20140-3/FIPS-140-3-CMVP%20Management%20Manual.pdf)) process — a simpler FIPS re-certification that typically reduces the waiting time from 12–18 months down to 3–4 months. Compared to reusing the binary entirely, this option requires less engineering effort, since engineers need only maintain source-level API compatibility (i.e., by patching the main kernel source code) rather than binary-level ABI compatibility between the crypto module and the main kernel. + +In summary, converting the kernel crypto subsystem into a loadable module enables reuse of a certified module across kernel updates. Whether through binary reuse, source-code reuse, or fresh certification, different choices represent different tradeoffs, and distributions can balance crypto feature availability, certification turnaround time, and engineering effort according to their needs. + +--- +Written by Jay Wang <[email protected]> <[email protected]>, Amazon Linux
Signature · 1 line
2.47.3