mm/vmalloc: make vm_struct.nr_pages an unsigned long

A vmalloc area can hold more than 2^32 pages, but the page counts tracking it are 32 bit. That has produced the same truncation bug twice already, once in vread_iter() and once in the vrealloc() grow-in-place check, and it keeps the casts that hide it scattered around the file.

Widen the field and everything that feeds or consumes it, so the byte counts derived from it are computed in 64 bit arithmetic without a cast at each site:

- vm_area_alloc_pages() takes and returns a page count, and the result is stored into nr_pages, so its parameter, return type and the nr_allocated and nr_remaining accumulators widen too. The 100 page cap on a bulk request stays narrow, only its literal becomes 100UL.

- nr_small_pages in __vmalloc_area_node() is derived from a 64 bit size and compared against nr_pages, so it widens as well. Before this the store truncated for a size of 16 TiB or more, which sized area->pages from the wrapped count while __vmap_pages_range() still walked the full size. The cast on array_size next to it was already redundant and goes away with it.

- new_nr_pages and old_nr_pages in vrealloc_node_align_noprof() widen, which drops the four casts on them, and the casts added by the two preceding patches in vrealloc_node_align_noprof() and vread_iter() are no longer needed either.

- vm_area_free_pages() takes a page index range.

Three loop counters that index area->pages were plain int and would have overflowed at 2^31 pages: in set_area_direct_map(), in vm_reset_perms() and in the bulk allocation loop, where the index is seeded from nr_allocated. They become unsigned long.

Printing needed fixing in two places. vmalloc_dump_obj() used %u, and vmalloc_info_show() printed the unsigned field with %d, which would have shown a negative page count for an area of 8 TiB or more. show_numa_info() used a single unsigned int as both the page index and the node id, so the page loop gets its own unsigned long counter.

The page table walkers are not covered here. vmap_pages_pte_range() and the levels above it carry the page cursor as int *nr, so a mapping is still capped independently of the counters this patch widens. Widening that cursor touches the whole walker and is separate work.

Nothing outside mm/vmalloc.c needs a change to build or to behave as before. Those users either index the pages array or multiply the count by PAGE_SIZE, which is unsigned long already. kexec_handover stores the count into a 32 bit field of its own ABI, which caps that interface exactly where it did before.

On x86-64 this leaves sizeof(struct vm_struct) at 72 bytes when CONFIG_HAVE_ARCH_HUGE_VMALLOC=n, since the existing padding before phys_addr absorbs the change, and takes it to 80 when the config is enabled. Both sizes come from the same kmalloc-96 bucket that __get_vm_area_node() allocates from, so the footprint per area does not change either way.

---
 include/linux/vmalloc.h |  2 +-
 mm/vmalloc.c            | 62 ++++++++++++++++++++---------------------
 2 files changed, 31 insertions(+), 33 deletions(-)

index d87dc7f77f4e8..a39c20c77efcd 100644
@@ -62,7 +62,7 @@ struct vm_struct {
 #ifdef CONFIG_HAVE_ARCH_HUGE_VMALLOC
 	unsigned int		page_order;
 #endif
-	unsigned int		nr_pages;
+	unsigned long		nr_pages;
 	phys_addr_t		phys_addr;
 	const void		*caller;
 	unsigned long		requested_size;

base-commit: 248951ddc14de84de3910f9b13f51491a8cd91df
prerequisite-patch-id: bc462c518eb9ddcaccad5e1f5fed7cc24512b117
prerequisite-patch-id: 891688c2e93c570d1c9c7802b46aaba974002684

2.43.0

I had a "wait, what, really?" moment. And it can! 2^32 pages is 16TiB, and x86-64 says:

ffffc90000000000 | -55 TB | ffffe8ffffffffff | 32 TB | vmalloc/ioremap space (vmalloc_base)

So we can have _one_ allocation that is more than 2^32 pages. But I'm not sure where we'll get the memory to populate it from.
https://blogs.oracle.com/cloud-infrastructure/announcing-oci-compute-e6-acceleron-instances
says I can get 2.3TB of memory in a single machine, which is a factor of 8 away from where we'd need to be to actually hit this.

Intel are adveertising a max memory of 1.5TiB on the 6990E+ (128GiB * 12 channels). So if you put 8 Xeons in a box and loaded it up, that could get you to 12TiB.

I'm not saying we shouldn't fix this, but it's all theoretical for now, right?

Artem said "That has produced the same truncation bug twice already, once in vread_iter() and once in the vrealloc() grow-in-place check". This might imply that the issue has been hit in practice, unclear.

Anyway, it's a good change so let's do it.

Sashiko wasn't able to apply it, and the mechanical checking would be useful here. I was able to get it to apply, took a bit of work.

So Artem, can you please update the changelog to clarify the above, redo against mm.git's mm-new branch (or linux-next) and send us a v2?

The 2^32 page part is, yes, and my opening sentence pointed at the wrong thing.

What went wrong twice wasn't a page count, it was a byte count. nr_pages is unsigned int, so nr_pages << PAGE_SHIFT is 32-bit arithmetic and wraps at 4GiB of bytes, which is 2^20 pages. In vread_iter() that made /proc/kcore hand back zeros for a 4GiB area as a successful read, and any machine with more than 4GiB of memory can allocate one. So the reachable part is 4GiB and the 16TiB framing was me leading with the wrong number.

To answer the other half of it, and Andrew's question too: no report behind any of this, I found it reading the code. v2 says so in as many words.

Artem

Andrew asked for a v2 of the nr_pages widening, rebased on mm-new, with the changelog clarified on what has actually been observed.

The widening reverts the two casts added by the patches I sent on 25 July, so it needs those in the tree first, and they are not in mm-new, mm-unstable or mm-hotfixes-unstable today. They are here as 1/3 and 2/3 so that the series applies and Sashiko can chew on it. 2/3 is byte for byte what I sent; 1/3 has one corrected sentence in its changelog and the same one-line diff.

Keeping the three separate is deliberate. 1/3 is a one-line fix for a v7.2 regression and carries a Fixes: tag, so it is the part that wants to go in on its own. 3/3 is a type change across the file.

On what has been seen in practice, since v1 of 3/3 read as though the bug had been hit: it has not. There is no report behind any of this, I found it reading the code. What is reachable is the 4 GiB truncation in vread_iter(), which needs a machine with more than 4 GiB of memory and nothing else, and makes /proc/kcore return zeros for such an area while reporting a successful read. The vrealloc() case has the same cause but stays latent, no in-tree caller grows an allocation that far. The rest is unexercised, and 3/3 now separates the two thresholds instead of lumping them together: 2^32 pages, where the field truncates, is 16 TiB and out of reach of any hardware, while 2^31, where the plain int page indexes break, is 8 TiB and only out of reach because nothing asks for an area that large.

Changes in v2:
- rebased on mm-new
- 3/3: the changelog separates what is reachable, what is latent and what is merely unexercised, and answers the 16 TiB question directly
- 3/3: the note on the mapping path no longer calls the int *nr cursor a cap, since it is an overflow and not a bound, and no longer claims that every user outside mm/vmalloc.c only indexes or shifts the count. Three of them pass it to a narrower parameter, which is now named
- 1/3: dropped the claim that alloc_large_system_hash() cannot reach a 4 GiB area. Its cap is a sixteenth of memory, so it can on a 64 GiB machine, just not without an explicit table size on the command line

v1 of 3/3:
https://lore.kernel.org/linux-mm/[email protected]/
1/3 and 2/3 as first posted:
https://lore.kernel.org/linux-mm/[email protected]/

Checked on x86-64: vmlinux and modules build clean, and mm/vmalloc.c plus every file that touches vm_struct::nr_pages are clean at W=1, with HAVE_ARCH_HUGE_VMALLOC, NUMA, KEXEC_HANDOVER, DEBUG_KMEMLEAK, DMA_API_DEBUG and PROC_KCORE enabled. sizeof(struct vm_struct) measured against the real headers: 72 bytes before and after with HAVE_ARCH_HUGE_VMALLOC=n, 72 to 80 with it enabled, both sizes inside the kmalloc-96 bucket that __get_vm_area_node() already allocates from.

Artem Lytkin (3):
mm/vmalloc: fix 32-bit truncation of the area size in vread_iter() mm/vmalloc: fix 32-bit truncation in the vrealloc() grow-in-place
check
mm/vmalloc: make vm_struct.nr_pages an unsigned long

 include/linux/vmalloc.h |  2 +-
 mm/vmalloc.c            | 58 ++++++++++++++++++++---------------------
 2 files changed, 29 insertions(+), 31 deletions(-)

base-commit: eee677bbc48890b2bcaa42ea7942478302937a09

2.43.0

vm_struct::nr_pages is an unsigned int, and the file keeps deriving byte counts from it as nr_pages << PAGE_SHIFT. A shift is evaluated in the type of its promoted left operand, so those are 32-bit arithmetic and wrap at 4 GiB of bytes, which is 2^20 pages. Every site depends on a cast being remembered; vmap() has one, two recent commits did not. vread_iter() then computes a size of zero for a 4 GiB VM_ALLOC area and /proc/kcore returns it as zeros while reporting a successful read, which drgn, crash or gdb cannot tell from real memory, and the vrealloc() grow-in-place check declines a request that would have fit.

Widen the field so the class of bug goes away instead of one site at a time. Everything feeding or consuming it widens too: vm_area_alloc_pages() and its accumulators, nr_small_pages, new_nr_pages and old_nr_pages, the index range of vm_area_free_pages(), and three page indexes that were plain int. Five casts go. Two prints needed fixing as well, %u in vmalloc_dump_obj() and %d for the unsigned field in vmalloc_info_show().

No bug report behind this, I found it reading the code. The 4 GiB wrap needs only a machine with over 4 GiB of memory. Neither larger threshold is a practical concern: 2^32 pages, where the field itself truncates, is 16 TiB and beyond what hardware can populate, and 2^31, where the plain int indexes break, is 8 TiB and larger than anything in the tree asks for. The int *nr cursor in the mapping path is unchanged and is separate work. Users outside mm/vmalloc.c need no change either; those handing the count to a narrower parameter cannot drive it near 2^31, and kexec_handover still caps its own 32-bit ABI field where it did.

sizeof(struct vm_struct) on x86-64 stays 72 bytes with CONFIG_HAVE_ARCH_HUGE_VMALLOC=n and goes from 72 to 80 with it enabled, both inside the kmalloc-96 bucket it already comes from.

---
v3:
- collapsed to a single patch. v2 put the two cast fixes ahead of this one so the series would apply, and this one then undid them; widening the field fixes both sites by itself (Uladzislau)
- shorter changelog (Uladzislau)
v2: https://lore.kernel.org/linux-mm/[email protected]/
v1: https://lore.kernel.org/linux-mm/[email protected]/

 include/linux/vmalloc.h |  2 +-
 mm/vmalloc.c            | 58 ++++++++++++++++++++---------------------
 2 files changed, 29 insertions(+), 31 deletions(-)

index e4d8d0a9f30f9..aed121d729b01 100644
@@ -62,7 +62,7 @@ struct vm_struct {
 #ifdef CONFIG_HAVE_ARCH_HUGE_VMALLOC
 	unsigned int		page_order;
 #endif
-	unsigned int		nr_pages;
+	unsigned long		nr_pages;
 	phys_addr_t		phys_addr;
 	const void		*caller;
 	unsigned long		requested_size;

base-commit: eee677bbc48890b2bcaa42ea7942478302937a09

2.43.0

Sashiko couldn't apply it for some reason.

This all seems far too complex for such an obscure issue.

How about we queue just a single switch-to-ulong patch, add that to 7.3 and give it a Fixes:0bca23804632/cc:stable so 7.2.x gets fixed later?

vm_struct::nr_pages is an unsigned int, and the file keeps deriving byte counts from it as nr_pages << PAGE_SHIFT. A shift is evaluated in the type of its promoted left operand, so those are 32-bit arithmetic and wrap at 4 GiB of bytes, which is 2^20 pages. Every site depends on a cast being remembered; vmap() has one, two recent commits did not. vread_iter() then computes a size of zero for a 4 GiB VM_ALLOC area and /proc/kcore returns it as zeros while reporting a successful read, which drgn, crash or gdb cannot tell from real memory, and the vrealloc() grow-in-place check declines a request that would have fit.

Widen the field so the class of bug goes away instead of one site at a time. Everything feeding or consuming it widens too: vm_area_alloc_pages() and its accumulators, nr_small_pages, new_nr_pages and old_nr_pages, the index range of vm_area_free_pages(), and three page indexes that were plain int. Five casts go. Two prints needed fixing as well, %u in vmalloc_dump_obj() and %d for the unsigned field in vmalloc_info_show().

No bug report behind this, I found it reading the code. The 4 GiB wrap needs only a machine with over 4 GiB of memory. Neither larger threshold is a practical concern: 2^32 pages, where the field itself truncates, is 16 TiB and beyond what hardware can populate, and 2^31, where the plain int indexes break, is 8 TiB and larger than anything in the tree asks for. The int *nr cursor in the mapping path is unchanged and is separate work. Users outside mm/vmalloc.c need no change either. Those handing the count to a narrower parameter cannot drive it near 2^31, and kho_preserve_vmalloc() stores it into a 32-bit ABI field that still receives the same low bits; above 2^32 pages the truncation just moves out of vm_struct into that store.

sizeof(struct vm_struct) on x86-64 stays 72 bytes with CONFIG_HAVE_ARCH_HUGE_VMALLOC=n and goes from 72 to 80 with it enabled, both inside the kmalloc-96 bucket it already comes from.

---
This is the single switch-to-ulong patch you asked for; v3 crossed with your mail a few hours earlier and was already that, only without the stable tag. So v4 is v3 plus Cc: stable, rebased on today's mm-new.

I left the second Fixes: on d57ac904ffdc as well, since the same widening is what fixes the vrealloc() grow-in-place check. Drop it if you would rather the backport hang off one commit.

v4:
- Cc: stable (Andrew)
- corrected the kexec_handover sentence, which claimed a cap that is not there; the ABI field just keeps taking the low 32 bits
- rebased on mm-new
v3: https://lore.kernel.org/linux-mm/[email protected]/
v2: https://lore.kernel.org/linux-mm/[email protected]/

 include/linux/vmalloc.h |  2 +-
 mm/vmalloc.c            | 58 ++++++++++++++++++++---------------------
 2 files changed, 29 insertions(+), 31 deletions(-)

index e4d8d0a9f30f9..aed121d729b01 100644
@@ -62,7 +62,7 @@ struct vm_struct {
 #ifdef CONFIG_HAVE_ARCH_HUGE_VMALLOC
 	unsigned int		page_order;
 #endif
-	unsigned int		nr_pages;
+	unsigned long		nr_pages;
 	phys_addr_t		phys_addr;
 	const void		*caller;
 	unsigned long		requested_size;

base-commit: 1dbd7c34bb92dd9c0b0363b75f6ec444299af108

2.43.0

Thanks.

Ulad, AI review suggests that vrealloc() has an issue handling __GFP_ZERO. Can you please check?

	https://sashiko.dev/#/patchset/[email protected]

I have checked. I think the AI is missing at least one point. AI argument which is:

<snip>
If a driver initially allocates memory using vmalloc() without __GFP_ZERO (leaving spare page capacity uninitialized), and then grows the allocation using vrealloc() with __GFP_ZERO, the caller expects the newly exposed bytes to be zeroed.
<snip>

In the vrealloc_node_align_noprof() header documentation there is a statement:

<snip>
* If __GFP_ZERO logic is requested, callers must ensure that, starting with the
* initial memory allocation, every subsequent call to this API for the same
* memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible that
* __GFP_ZERO is not fully honored by this API.
<snip>

AI argument violates the documentation, i.e. mixing __GFP_ZERO is not allowed.

From the other hand we can mix it and remove that part of documentation:

<snip>

index 7a0cbba3d29d..28d0fed94d22 100644
@@ -4294,11 +4294,6 @@ EXPORT_SYMBOL(vzalloc_node_noprof);
  * __GFP_THISNODE flag should be set, otherwise the function will try to avoid
  * reallocation and possibly disregard the specified @nid.
  *
- * If __GFP_ZERO logic is requested, callers must ensure that, starting with the
- * initial memory allocation, every subsequent call to this API for the same
- * memory allocation is flagged with __GFP_ZERO. Otherwise, it is possible that
- * __GFP_ZERO is not fully honored by this API.
- *
  * Requesting an alignment that is bigger than the alignment of the existing
  * allocation will fail.
  *

@@ -4415,13 +4410,12 @@ void *vrealloc_node_align_noprof(const void *p, size_t size, unsigned long align
 	 * We already have the bytes available in the allocation; use them.
 	 */
 	if (size <= vm->nr_pages << PAGE_SHIFT) {
-		/*
-		 * No need to zero memory here, as unused memory will have
-		 * already been zeroed at initial allocation time or during
-		 * realloc shrink time.
-		 */
-		vm->requested_size = size;
 		kasan_vrealloc(p, old_size, size);
+
+		if (want_init_on_alloc(flags))
+			memset((void *)p + old_size, 0, size - old_size);
+
+		vm->requested_size = size;
 		return (void *)p;
 	}

<snip>

--
Uladzislau Rezki

Sashiko is talking about the initial allocation not using __GFP_ZERO but vrealloc() *does* use __GFP_ZERO. The documentation you quoted doesn't address that case?

Also, developers don't read documentation ;) What happens if some caller *does* use vmalloc(!__GFP_ZERO) then vrealloc(__GFP_ZERO)? Silent misbehavior would be bad - it would be good if vrealloc() were to drop a WARN() then ignore the __GFP_ZERO.

But this is not allowed according to doc :)

<snip>
...
callers must ensure that, starting with the initial memory allocation
...
<snip>

Also, AI describes the situation like:

vmalloc()
vrealloc(grow, since need more)

i.e. from the description:
<snip>
and then grows the allocation using vrealloc() with __GFP_ZERO, the caller expects the newly exposed bytes to be zeroed.
<snip>

and it will be zeroed in fact, because the path would be:

<snip>
need_realloc:

	/* TODO: Grow the vm_area, i.e. allocate and map additional pages. */
	n = __vmalloc_node_noprof(size, align, flags, nid, __builtin_return_address(0));

	if (!n)
		return NULL;

<snip>

and not the one which AI pointed to.

The real scenario is:

1. vmalloc(!GFP_ZERO) - alloc size 10
2. vrealloc(!GFP_ZERO) - realloc to size 5
3. vrealloc(GFP_ZERO) - realloc back to 10.

3 - will not zeroed. As noted in doc ZERO should be used starting from the beginning [1]. But in __most__ cases it will be zeroed anyway because we free/unmap tail pages and if:

<snip>
/*
* Free tail pages when shrink crosses a page boundary. *
* Skip huge page allocations (page_order > 0) as partial
* freeing would require splitting.
*
* Skip VM_FLUSH_RESET_PERMS, as direct-map permissions must
* be reset before pages are returned to the allocator. *
* Skip VM_USERMAP, as remap_vmalloc_range_partial() validates
* mapping requests against the unchanged vm->size; freeing
* tail pages would cause vmalloc_to_page() to return NULL for
* the unmapped range.
*
* Skip if either GFP_NOFS or GFP_NOIO are used.
* kmemleak_free_part() internally allocates with
* GFP_KERNEL, which could trigger a recursive deadlock
* if we are under filesystem or I/O reclaim.
*/
if (new_nr_pages < vm->nr_pages && !vm_area_page_order(vm) &&
!(vm->flags & (VM_FLUSH_RESET_PERMS | VM_USERMAP)) && gfp_has_io_fs(flags)) {
<snip>

is true the next grow with GFP_ZERO will be zeroed.

For huge alloc it will not be zeroed and for other conditions.

OK. I will prepare something and send out the patch after testing.

--
Uladzislau Rezki

Commit 0bca23804632 ("mm/vmalloc: use physical page count in vread_iter() for VM_ALLOC areas") replaced get_vm_area_size(vm), which returns a size_t, with vm->nr_pages << PAGE_SHIFT.

struct vm_struct::nr_pages is an unsigned int. The shift operator does not perform the usual arithmetic conversions: the integer promotions are applied to each operand and the type of the result is that of the promoted left operand. The expression is therefore evaluated in 32-bit arithmetic no matter how PAGE_SHIFT is typed, and no matter that the result is assigned to a size_t. Once an area reaches 4 GiB the byte count wraps, at 1 << 20 pages with 4 KiB pages, 1 << 18 with 16 KiB and 1 << 16 with 64 KiB.

nr_pages counts base pages even for huge vmalloc mappings, since __vmalloc_area_node() requires area->nr_pages == nr_small_pages, so a huge page_order does not raise the threshold.

The only limit on the size of a single area is the totalram_pages() check in __vmalloc_node_range_noprof(), so any machine with more than 4 GiB of memory can create an affected area. One example is the zram metadata table, which is a single vzalloc() of disksize / 256 with CONFIG_LOCKDEP=n: "echo 1T > /sys/block/zram0/disksize" allocates exactly 4 GiB and needs no 1 TiB of anything. Sufficiently large BPF array or prealloc-hash maps do it too, as does
"modprobe test_vmalloc run_test_mask=1 nr_pages=1048576". The KVM dirty bitmap is one path that might be expected to reach it but cannot, being bounded by KVM_MEM_MAX_NR_PAGES to 512 MiB. alloc_large_system_hash() can reach it, but not on its own: it caps a table at a sixteenth of memory, which is 4 GiB once a machine has 64 GiB, and the automatic sizing stays orders of magnitude below that cap, so it takes an explicit table size on the command line.

The effect is confined to /proc/kcore, the only caller. For an area whose size is an exact multiple of 4 GiB the computed size becomes 0, the

	if (addr >= vaddr + size)
		goto next_va;

test succeeds and the whole area is skipped; for other sizes only the first nr_pages mod 2^20 pages are read and the rest is skipped. Either way the bytes are zero-filled at the finished_zero label, which returns the full requested length, so read_kcore_iter() sees a successful read and userspace gets neither an error nor a short read. Live inspection with drgn, crash or gdb silently observes zeros where the area is populated, and cannot distinguish that from genuinely zeroed memory. Before the offending commit the size came from vm->size in 64-bit arithmetic, so this is a v7.2 regression.

The truncated value is always less than or equal to the true size, so vread_iter() can only under-read; there is no out-of-bounds access. Both the affected reader and every producer above are privileged: opening /proc/kcore requires CAP_SYS_RAWIO and is refused under lockdown. This is a correctness and debuggability problem, not a security one.

Fix it by widening the shift, which also makes the expression consistent with the four (unsigned long)nr_pages << PAGE_SHIFT expressions in vrealloc_node_align_noprof().

On 32-bit a widening cast cannot help, size_t being 32 bits there as well, but a 4 GiB vmalloc area is not reachable on 32-bit either. On 64-bit the cast removes the truncation entirely, which is why replacing get_vm_area_size() introduced a regression rather than inheriting a pre-existing wart.

---
 mm/vmalloc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 26f32949c2f2e..34e10b825889a 100644
@@ -4895,7 +4895,7 @@ long vread_iter(struct iov_iter *iter, const char *addr, size_t count)
 			 * mapping types (vmap, ioremap) don't set nr_pages.
 			 */
 			size = (vm->flags & VM_ALLOC && vm->nr_pages) ?
-				       (vm->nr_pages << PAGE_SHIFT) :
+				       ((unsigned long)vm->nr_pages << PAGE_SHIFT) :
 				       get_vm_area_size(vm);
 		else
 			size = va_size(va);

2.43.0

Commit d57ac904ffdc ("mm/vmalloc: use physical page count for vrealloc() grow-in-place check") introduced

	if (size <= vm->nr_pages << PAGE_SHIFT) {

in place of a comparison against a size_t. As vm->nr_pages is an unsigned int and a shift is evaluated in the type of its promoted left operand, the right hand side is computed in 32-bit arithmetic and wraps for areas of 4 GiB or more. Twelve lines above, in the same function, four expressions cast for exactly this reason:

	kmemleak_free_part((void *)addr + ((unsigned long)new_nr_pages
					   << PAGE_SHIFT), ...);

The consequence here is benign. Truncation only ever makes the right hand side smaller than the real capacity, so the test can only fail when it should have succeeded: vrealloc() then falls through to need_realloc, allocates a new area, copies min(size, old_size) bytes and frees the old one. That is correct, only wasteful, and no in-tree caller currently grows an allocation past 4 GiB, so this is a latent fix rather than a user-visible one.

Widen the shift to match the surrounding code. Note that comparing page counts instead, as in PAGE_ALIGN(size) >> PAGE_SHIFT <= vm->nr_pages, would avoid the cast but introduce a real bug: PAGE_ALIGN() wraps to 0 for sizes within a page of ULONG_MAX and the test would then wrongly succeed.

---
 mm/vmalloc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index 34e10b825889a..bf9e32cf93fc9 100644
@@ -4588,7 +4588,7 @@ void *vrealloc_node_align_noprof(const void *p, size_t size, unsigned long align
 	/*
 	 * We already have the bytes available in the allocation; use them.
 	 */
-	if (size <= vm->nr_pages << PAGE_SHIFT) {
+	if (size <= (unsigned long)vm->nr_pages << PAGE_SHIFT) {
 		/*
 		 * No need to zero memory here, as unused memory will have
 		 * already been zeroed at initial allocation time or during

2.43.0

vm_struct::nr_pages is an unsigned int, and the file repeatedly derives a byte count from it as nr_pages << PAGE_SHIFT. A shift is evaluated in the type of its promoted left operand, so those expressions are 32-bit arithmetic and wrap at 4 GiB of bytes, which is only 2^20 pages with 4 KiB pages. Every such site therefore depends on someone remembering a cast. vmap() has one. The two commits fixed by the preceding two patches did not.

v1 of this patch was ambiguous about how much of that has been seen in practice, so to be explicit: none of it comes from a bug report, the series comes out of reading the code. What is reachable on current hardware is the 4 GiB wrap. A single vzalloc() of 4 GiB needs a machine with more than 4 GiB of memory and nothing else, and /proc/kcore then reads such an area as zeros while reporting a successful read. The vrealloc() case has the same cause, but no in-tree caller grows an allocation that far, so it stays latent. The two thresholds above that are not equally far away. A count above 2^32, where the field itself truncates, is out of reach: that is 16 TiB, and as Matthew Wilcox notes, current hardware cannot populate it, topping out near 12 TiB with eight fully loaded sockets. The 2^31 point, where the plain int page indexes below stop working, is 8 TiB, which such a machine could in principle hand to one allocation, since the only size gate on a request is the totalram_pages() check in __vmalloc_node_range_noprof(). No in-tree caller comes near it, so that one is unexercised rather than impossible.

Widening the field is still worth doing, because it retires the 4 GiB class of bug at the type level instead of one call site at a time, and the casts go with it.

Widen the field and everything that feeds or consumes it:

- vm_area_alloc_pages() takes and returns a page count that is stored into nr_pages, so its parameter, its return type and the nr_allocated and nr_remaining accumulators widen with it. The 100 page cap on a bulk request stays narrow, only its literal becomes 100UL.

- nr_small_pages in __vmalloc_area_node() is derived from a 64-bit size and compared against nr_pages, so it widens too. Before this the store truncated at 16 TiB, which sized area->pages from the wrapped count while the mapping still covered the full size. The cast on array_size next to it was already redundant and goes away with it.

- new_nr_pages and old_nr_pages in vrealloc_node_align_noprof() widen, which drops four casts, and the casts added by the two preceding patches are no longer needed either.

- vm_area_free_pages() takes a page index range.

Three counters that index area->pages were plain int and would have overflowed at 2^31 pages: in set_area_direct_map(), in vm_reset_perms() and in the bulk allocation loop, where the index is seeded from nr_allocated. They become unsigned long.

Two prints needed fixing. vmalloc_dump_obj() used %u, and vmalloc_info_show() printed the unsigned field with %d, which would have shown a negative page count. show_numa_info() used a single unsigned int as both the page index and the node id, so the page loop gets its own counter.

The mapping path is deliberately left alone. vmap_pages_pte_range() and the levels above it carry the page cursor as int *nr, seeded in vmap_pages_range_noflush_walk(). That is not a cap: past 2^31 base pages it overflows and indexes area->pages with a negative value. It is also not made worse here, because an unsigned int nr_pages already spanned 2^31 to 2^32-1, and nothing about a request's size was ever gated on the width of that cursor. Widening it touches every level of the walker and is separate work.

Nothing outside mm/vmalloc.c needs a change to build or to behave as before. Most of those users index the pages array or shift the count by PAGE_SHIFT, which is unsigned long arithmetic either way. Three hand it to a narrower parameter: set_memory_rox() in execmem_cache_populate_alloc() and release_pages() in free_mod_tags_mem() take an int page count, and clear_pages() in alloc_thread_stack_node() takes an unsigned int. Two of those already narrowed an unsigned int, and none of the three can approach 2^31: a thread stack is THREAD_SIZE / PAGE_SIZE pages, and an execmem block and the module tag area are both far smaller. kexec_handover stores the count into a 32-bit field of its own ABI, which caps that interface exactly where it did before.

On x86-64 sizeof(struct vm_struct) stays 72 bytes with CONFIG_HAVE_ARCH_HUGE_VMALLOC=n, since the existing padding before phys_addr absorbs the change, and becomes 80 with it enabled. Both sizes come from the same kmalloc-96 bucket that __get_vm_area_node() allocates from, so the footprint per area does not change either way.

---
 include/linux/vmalloc.h |  2 +-
 mm/vmalloc.c            | 62 ++++++++++++++++++++---------------------
 2 files changed, 31 insertions(+), 33 deletions(-)

index e4d8d0a9f30f9..aed121d729b01 100644
@@ -62,7 +62,7 @@ struct vm_struct {
 #ifdef CONFIG_HAVE_ARCH_HUGE_VMALLOC
 	unsigned int		page_order;
 #endif
-	unsigned int		nr_pages;
+	unsigned long		nr_pages;
 	phys_addr_t		phys_addr;
 	const void		*caller;
 	unsigned long		requested_size;

2.43.0

Makes sense to me. The commit message could be shorter describing just an issue. I was lost when i read it first time.

Also:

<snip>
Artem Lytkin (3):
mm/vmalloc: fix 32-bit truncation of the area size in vread_iter() mm/vmalloc: fix 32-bit truncation in the vrealloc() grow-in-place check mm/vmalloc: make vm_struct.nr_pages an unsigned long
<snip>

first two fix overlaps. The latest also do it by converting nr_pages to unsigned long and removes (unsigned long) casting introduced by 1 and 2.

Maybe just use one commit which is last in this series?

Anyway for this

--
Uladzislau Rezki