mm/memblock: Add "reserve_mem" to reserved named memory at boot up
Subsystems (6)
Steven Rostedt
maintainerv5 cover mm/memblock: Add "reserve_mem" to reserved named memory at boot up
Reserve unspecified location of physical memory from kernel command line
Background:
In ChromeOS, we have 1 MB of pstore ramoops reserved so that we can extract dmesg output and some other information when a crash happens in the field. (This is only done when the user selects "Allow Google to collect data for
improving the system"). But there are cases when there's a bug that
requires more data to be retrieved to figure out what is happening. We would like to increase the pstore size, either temporarily, or maybe even permanently. The pstore on these devices are at a fixed location in RAM (as the RAM is not cleared on soft reboots nor crashes). The location is chosen by the BIOS (coreboot) and passed to the kernel via ACPI tables on x86. There's a driver that queries for this to initialize the pstore for ChromeOS:
See drivers/platform/chrome/chromeos_pstore.c
Problem:
The problem is that, even though there's a process to change the kernel on these systems, and is done regularly to install updates, the firmware is updated much less frequently. Choosing the place in RAM also takes special care, and may be in a different address for different boards. Updating the size via firmware is a large effort and not something that many are willing to do for a temporary pstore size change.
Requirement:
Need a way to reserve memory that will be at a consistent location for every boot, if the kernel and system are the same. Does not need to work if rebooting to a different kernel, or if the system can change the memory layout between boots.
The reserved memory can not be an hard coded address, as the same kernel / command line needs to run on several different machines. The picked memory reservation just needs to be the same for a given machine, but may be different for different machines.
Solution:
The solution I have come up with is to introduce a new "reserve_mem=" kernel command line. This parameter takes the following format:
reserve_mem=nn:align:label
Where nn is the size of memory to reserve, the align is the alignment of that memory, and label is the way for other sub-systems to find that memory. This way the kernel command line could have:
reserve_mem=12M:4096:oops ramoops.mem_name=oops
At boot up, the kernel will search for 12 megabytes in usable memory regions with an alignment of 4096. It will start at the highest regions and work its way down (for those old devices that want access to lower address DMA). When it finds a region, it will save it off in a small table and mark it with the "oops" label. Then the pstore ramoops sub-system could ask for that memory and location, and it will map itself there.
This prototype allows for 8 different mappings (which may be overkill, 4 is probably plenty) with 16 byte size to store the label.
I have tested this and it works for us to solve the above problem. We can update the kernel and command line and increase the size of pstore without needing to update the firmware, or knowing every memory layout of each board. I only tested this locally, it has not been tested in the field.
Changes since v4: https://lore.kernel.org/all/[email protected]/
[ patch at bottom showing differences ]
- Add all checks about reserve_mem before allocation. This means reserved_mem_add() is now a void function.
- Check for name duplications.
- Fix compare of align to SMP_CACHE_BYTES ("<" instead of "<=")
Changes since v3: https://lore.kernel.org/all/[email protected]/
- Changed table type of start and size from unsigned long to phys_addr_t (as well as the parameters to the functions that use them)
- Changed old reference to "early_reserve_mem" to "reserve_mem"
- Check before reservering memory:
o Size is non-zero
o name has text in it
- If align is less than SMP_CACHE_BYTES, make it SMP_CACHE_BYTES
- Remove the silly check of testing *p == '\0' after a p += strlen(p)
Changes since v2: https://lore.kernel.org/all/[email protected]/
- Fixed typo of "reserver"
- Added EXPORT_SYMBOL_GPL() for reserve_mem_find_by_name()
- Removed "built-in" from module description that was changed from v1.
Changes since v1: https://lore.kernel.org/all/[email protected]/
- Updated the change log of the first patch as well as added an entry into kernel-parameters.txt about how reserve_mem is for soft reboots and may not be reliable.
Steven Rostedt (Google) (2):
mm/memblock: Add "reserve_mem" to reserved named memory at boot up pstore/ramoops: Add ramoops.mem_name= command line option
----
Documentation/admin-guide/kernel-parameters.txt | 20 ++++ fs/pstore/ram.c | 14 +++ include/linux/mm.h | 2 + mm/memblock.c | 117 ++++++++++++++++++++++++ 4 files changed, 153 insertions(+)
mm/memblock.c +18 -16
@@ -2256,22 +2256,15 @@ static struct reserve_mem_table reserved_mem_table[RESERVE_MEM_MAX_ENTRIES];static int reserved_mem_count;/* Add wildcard region with a lookup name */ -static int __init reserved_mem_add (phys_addr_t start,phys_addr_t size, +static void __init reserved_mem_add (phys_addr_t start,phys_addr_t size,const char *name) {struct reserve_mem_table *map; -if (!name || !name[0 ] ||strlen (name) >=RESERVE_MEM_NAME_SIZE ) -return -EINVAL ; - -if (reserved_mem_count >=RESERVE_MEM_MAX_ENTRIES ) -return -1 ; - map = &reserved_mem_table[reserved_mem_count++]; map->start = start; map->size = size;strscpy (map->name, name); -return 0 ; }/**
@@ -2280,6 +2273,8 @@ static int __init reserved_mem_add(phys_addr_t start, phys_addr_t size,* @start: If found, holds the start address * @size: If found, holds the size of the address. * +* @start and @size are only updated if @name is found. +* * Returns: 1 if found or 0 if not found. */ int reserve_mem_find_by_name (const char *name,phys_addr_t *start,phys_addr_t *size)
@@ -2309,11 +2304,15 @@ static int __init reserve_mem(char *p)phys_addr_t start, size, align;char *name;char *oldp; -int err; +int len;if (!p)return -EINVAL ; +/* Check if there's room for more reserved memory */ +if (reserved_mem_count >=RESERVE_MEM_MAX_ENTRIES ) +return -EBUSY ; + oldp = p; size =memparse (p, &p);if (!size || p == oldp)
@@ -2330,11 +2329,14 @@ static int __init reserve_mem(char *p)* memblock_phys_alloc() doesn't like a zero size align, * but it is OK for this command to have it. */ -if (align <=SMP_CACHE_BYTES ) +if (align <SMP_CACHE_BYTES ) align =SMP_CACHE_BYTES ; name = p +1 ; -if (!strlen (name)) + len =strlen (name); + +/* name needs to have length but not too big */ +if (!len || len >=RESERVE_MEM_NAME_SIZE )return -EINVAL ;/* Make sure that name has text */
@@ -2345,15 +2347,15 @@ static int __init reserve_mem(char *p)if (!*p)return -EINVAL ; +/* Make sure the name is not already used (size is only updated if found) */ +if (reserve_mem_find_by_name (name, &start, &size)) +return -EBUSY ; + start =memblock_phys_alloc (size, align);if (!start)return -ENOMEM ; - err =reserved_mem_add (start, size, name); -if (err) { -memblock_phys_free (start, size); -return err; - } +reserved_mem_add (start, size, name);return 0 ; }
Steven Rostedt
maintainerPATCH v5 1/2 mm/memblock: Add "reserve_mem" to reserved named memory at boot up
From: "Steven Rostedt (Google)" <[email protected]>
In order to allow for requesting a memory region that can be used for things like pstore on multiple machines where the memory layout is not the same, add a new option to the kernel command line called "reserve_mem".
The format is: reserve_mem=nn:align:name
Where it will find nn amount of memory at the given alignment of align. The name field is to allow another subsystem to retrieve where the memory was found. For example:
reserve_mem=12M:4096:oops ramoops.mem_name=oops
Where ramoops.mem_name will tell ramoops that memory was reserved for it via the reserve_mem option and it can find it by calling:
if (reserve_mem_find_by_name("oops", &start, &size)) {
// start holds the start address and size holds the size givenThis is typically used for systems that do not wipe the RAM, and this command line will try to reserve the same physical memory on soft reboots. Note, it is not guaranteed to be the same location. For example, if KASLR places the kernel at the location of where the RAM reservation was from a previous boot, the new reservation will be at a different location. Any subsystem using this feature must add a way to verify that the contents of the physical memory is from a previous boot, as there may be cases where the memory will not be located at the same location.
Not all systems may work either. There could be bit flips if the reboot goes through the BIOS. Using kexec to reboot the machine is likely to have better results in such cases.
---
Changes since v4: https://lore.kernel.org/linux-trace-kernel/[email protected]
- Add all checks about reserve_mem before allocation. This means reserved_mem_add() is now a void function.
- Check for name duplications.
- Fix compare of align to SMP_CACHE_BYTES ("<" instead of "<=")
.../admin-guide/kernel-parameters.txt | 20 +++ include/linux/mm.h | 2 + mm/memblock.c | 117 ++++++++++++++++++ 3 files changed, 139 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +20 -0
@@ -5710,6 +5710,26 @@ them. If <base> is less than 0x10000, the region is assumed to be I/O ports; otherwise it is memory. + reserve_mem= [RAM] + Format: nn[KNG]:<align>:<label> + Reserve physical memory and label it with a name that + other subsystems can use to access it. This is typically + used for systems that do not wipe the RAM, and this command + line will try to reserve the same physical memory on + soft reboots. Note, it is not guaranteed to be the same + location. For example, if KASLR places the kernel at the + location of where the RAM reservation was from a previous + boot, the new reservation will be at a different location. + Any subsystem using this feature must add a way to verify + that the contents of the physical memory is from a previous + boot, as there may be cases where the memory will not be + located at the same location. + + The format is size:align:label for example, to request + 12 megabytes of 4096 alignment for ramoops: + + reserve_mem=12M:4096:oops ramoops.mem_name=oops + reservetop= [X86-32,EARLY] Format: nn[KMG] Reserves a hole at the top of the kernel virtual
include/linux/mm.h +2 -0
@@ -4263,4 +4263,6 @@ static inline bool pfn_is_unaccepted_memory(unsigned long pfn)void vma_pgtable_walk_begin (struct vm_area_struct *vma);void vma_pgtable_walk_end (struct vm_area_struct *vma); +int reserve_mem_find_by_name (const char *name,phys_addr_t *start,phys_addr_t *size); +#endif /* _LINUX_MM_H */
mm/memblock.c +117 -0
123 lines not on this page · raw message
Signature · 1 line
2.43.0
Mike Rapoport
maintainerQuoted 57 lines · Steven Rostedt
From: "Steven Rostedt (Google)" <[email protected]>
In order to allow for requesting a memory region that can be used for things like pstore on multiple machines where the memory layout is not the same, add a new option to the kernel command line called "
reserve_mem".The format is: reserve_mem=nn:align:name
Where it will find nn amount of memory at the given alignment of align. The name field is to allow another subsystem to retrieve where the memory was found. For example:
reserve_mem=12M:4096:oops ramoops.mem_name=oops
Where ramoops.mem_name will tell ramoops that memory was reserved for it via the
reserve_memoption and it can find it by calling:if (reserve_mem_find_by_name("oops", &start, &size)) { // start holds the start address and size holds the size givenThis is typically used for systems that do not wipe the RAM, and this command line will try to reserve the same physical memory on soft reboots. Note, it is not guaranteed to be the same location. For example, if KASLR places the kernel at the location of where the RAM reservation was from a previous boot, the new reservation will be at a different location. Any subsystem using this feature must add a way to verify that the contents of the physical memory is from a previous boot, as there may be cases where the memory will not be located at the same location.
Not all systems may work either. There could be bit flips if the reboot goes through the BIOS. Using kexec to reboot the machine is likely to have better results in such cases.
Suggested-by Mike Rapoport <[email protected]>Tested-by Guilherme G. Piccoli <[email protected]>Signed-off-by Steven Rostedt (Google) <[email protected]>---
Changes since v4: https://lore.kernel.org/linux-trace-kernel/[email protected]- Add all checks about
reserve_membefore allocation. This meansreserved_mem_add()is now a void function.- Check for name duplications.
- Fix compare of align to
SMP_CACHE_BYTES("<" instead of "<=").../admin-guide/kernel-parameters.txt | 20 +++ include/linux/mm.h | 2 + mm/memblock.c | 117 ++++++++++++++++++ 3 files changed, 139 insertions(+)mm/memblock.c +0
-0
...
Quoted 44 lines
mm/memblock.c +44
-0+/* +* Parse reserve_mem=nn:align:name +*/ +static int __init reserve_mem (char *p) +{ +phys_addr_t start, size, align; +char *name; +char *oldp; +int len; + +if (!p) +return -EINVAL ; + +/* Check if there's room for more reserved memory */ +if (reserved_mem_count >=RESERVE_MEM_MAX_ENTRIES ) +return -EBUSY ; + + oldp = p; + size =memparse (p, &p); +if (!size || p == oldp) +return -EINVAL ; + +if (*p !=':' ) +return -EINVAL ; + + align =memparse (p+1 , &p); +if (*p !=':' ) +return -EINVAL ; + +/* +* memblock_phys_alloc() doesn't like a zero size align, +* but it is OK for this command to have it. +*/ +if (align <SMP_CACHE_BYTES ) + align =SMP_CACHE_BYTES ; + + name = p +1 ; + len =strlen (name); + +/* name needs to have length but not too big */ +if (!len || len >=RESERVE_MEM_NAME_SIZE ) +return -EINVAL ; + +/* Make sure that name has text */
mm/memblock.c void __init memblock_free_all(void) · line 2352
Quoted 10 lines
mm/memblock.c +10
-0+for (p = name; *p; p++) { +if (!isspace (*p)) +break ; + } +if (!*p) +return -EINVAL ; + +/* Make sure the name is not already used (size is only updated if found) */ +if (reserve_mem_find_by_name (name, &start, &size)) +return -EBUSY ;
I'd prefer another variable here rather than size. Will be more robust IMO.
Quoted 18 lines
mm/memblock.c +11
-0+ + start =memblock_phys_alloc (size, align); +if (!start) +return -ENOMEM ; + +reserved_mem_add (start, size, name); + +return 0 ; +} +__setup ("reserve_mem=" , reserve_mem); +#if defined(CONFIG_DEBUG_FS ) && defined(CONFIG_ARCH_KEEP_MEMBLOCK )static const char *const flagname[] = { [ilog2(MEMBLOCK_HOTPLUG)] = "HOTPLUG",Signature · 1 line
2.43.0
Signature · 2 lines
Sincerely yours, Mike.
Steven Rostedt
maintainerQuoted 5 lines · On Thu, 13 Jun 2024 09:00:31 +0300 Mike Rapoport <[email protected]> wrote:
mm/memblock.c void __init memblock_free_all(void) · line 2352
Quoted 3 lines
mm/memblock.c +3
-0+/* Make sure the name is not already used (size is only updated if found) */ +if (reserve_mem_find_by_name (name, &start, &size)) +return -EBUSY ;I'd prefer another variable here rather than size. Will be more robust IMO.
It's a damn habit of mine to be frugal on the stack variables. I'll update in v6.
-- Steve
Steven Rostedt
maintainerPATCH v5 2/2 pstore/ramoops: Add ramoops.mem_name= command line option
In reply to Steven RostedtFrom: "Steven Rostedt (Google)" <[email protected]>
Add a method to find a region specified by reserve_mem=nn:align:name for ramoops. Adding a kernel command line parameter:
reserve_mem=12M:4096:oops ramoops.mem_name=oops
Will use the size and location defined by the memmap parameter where it finds the memory and labels it "oops". The "oops" in the ramoops option is used to search for it.
This allows for arbitrary RAM to be used for ramoops if it is known that the memory is not cleared on kernel crashes or soft reboots.
--- fs/pstore/ram.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
fs/pstore/ram.c +14 -0
@@ -50,6 +50,10 @@ module_param_hw(mem_address, ullong, other, 0400);MODULE_PARM_DESC (mem_address,"start of reserved RAM used to store oops/panic logs" ); +static char *mem_name; +module_param_named (mem_name, mem_name, charp,0400 ); +MODULE_PARM_DESC (mem_name,"name of kernel param that holds addr" ); +static ulong mem_size;module_param (mem_size, ulong,0400 ); MODULE_PARM_DESC(mem_size,
@@ -914,6 +918,16 @@ static void __init ramoops_register_dummy(void) {struct ramoops_platform_data pdata; +if (mem_name) { +phys_addr_t start; +phys_addr_t size; + +if (reserve_mem_find_by_name (mem_name, &start, &size)) { + mem_address = start; + mem_size = size; + } + } +/* * Prepare a dummy platform data structure to carry the module * parameters. If mem_size isn't set, then there are no module
Signature · 1 line
2.43.0
Hi, Steven
Would you mind cc me in the following version?
Signature · 2 lines
Wei Yang Help you, Help me
Mike Rapoport
maintainerQuoted 3 lines · Wei Yang
Hi, Steven
Would you mind cc me in the following version?
v7: https://lore.kernel.org/all/[email protected]
Quoted 3 lines
Signature · 2 lines
Wei Yang Help you, Help me
Signature · 2 lines
Sincerely yours, Mike.