Skip to contentSkip to conversation

mm: return -ENOMEM for page-table allocation failure in insert_pages()

PATCH In review Avi Weiss 4 participants 7 messages Jul 30

Subsystems (2)
lore
Share & tools
lore

Avi Weiss

author
#0 lore raw

PATCH 1/1 mm: return -ENOMEM for page-table allocation failure in insert_pages()

walk_to_pmd() returns NULL only when p4d_alloc(), pud_alloc(), or pmd_alloc() fails. These are page-table allocation failures, but insert_pages() currently reports them as -EFAULT.

Return -ENOMEM instead, consistent with the subsequent pte_alloc() failure and with the single-page insert_page() path, which reports failure of the same page-table allocation chain as -ENOMEM.

Address and range validation failures in vm_insert_pages() continue to return -EFAULT. Keep the later -EFAULT return for pte_offset_map_lock(), which is not an allocation failure.

Fixes 8cd3984d81d5 ("mm/memory.c: add vm_insert_pages()")
---
 mm/memory.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/memory.c +1 -1
index ff338c2abe92..d8eddca8e251 100644
@@ -2436,7 +2436,7 @@ static int insert_pages(struct vm_area_struct *vma, unsigned long addr,
 	unsigned long pages_to_write_in_pmd;
 	int ret;
 more:
-	ret = -EFAULT;
+	ret = -ENOMEM;
 	pmd = walk_to_pmd(mm, addr);
 	if (!pmd)
 		goto out;
Signature · 1 line
2.43.0
#1 lore raw
Quoted 8 lines · Avi Weiss

walk_to_pmd() returns NULL only when p4d_alloc(), pud_alloc(), or pmd_alloc() fails. These are page-table allocation failures, but insert_pages() currently reports them as -EFAULT.

Return -ENOMEM instead, consistent with the subsequent pte_alloc() failure and with the single-page insert_page() path, which reports failure of the same page-table allocation chain as -ENOMEM.

Quoted 5 lines

Address and range validation failures in vm_insert_pages() continue to return -EFAULT. Keep the later -EFAULT return for pte_offset_map_lock(), which is not an allocation failure.

Fixes 8cd3984d81d5 ("mm/memory.c: add vm_insert_pages()")

Hmm :)

This isn't really a fix. Anybody relying on this returning -ENOMEM vs. -EFAULT here is in a state of sin anyway (unless you can point to specific users who are broken).

Drop the tag.

Quoted 1 line

This is correct, walk

Quoted 9 lines
---
 mm/memory.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/memory.c +0 -0
index ff338c2abe92..d8eddca8e251 100644

mm/memory.c static int insert_pages(struct vm_area_struct *vma, unsigned long addr, · line 2440

Quoted 6 lines
mm/memory.c +1 -1
@@ -2436,7 +2436,7 @@ static int insert_pages(struct vm_area_struct *vma, unsigned long addr,
 	unsigned long pages_to_write_in_pmd;
 	int ret;
 more:
-	ret = -EFAULT;
+	ret = -ENOMEM;
 	pmd = walk_to_pmd(mm, addr);

walk_to_pmd() is horribly named, it's allocating... populate_to_pmd() would be better can you rename it?

mm/memory.c static int insert_pages(struct vm_area_struct *vma, unsigned long addr, · line 2442

Quoted 2 lines
mm/memory.c +0 -0
 	if (!pmd)
 		goto out;

Looking down:

	pages_to_write_in_pmd = min_t(unsigned long,
		remaining_pages_total, PTRS_PER_PTE - pte_index(addr));

	/* Allocate the PTE if necessary; takes PMD lock once only. */
	ret = -ENOMEM; <------------------------------ set it again?
	if (pte_alloc(mm, pmd))
		goto out;

The way this function is written is horrible in general, I hate 'preset default return value' as a pattern.

So could you instead change it so the ret is set at the point of error, and while you're at it rename ret to err, e.g.:

	int err = 0;

	...

	pmd = populate_to_pmd(mm, addr);
	if (!pmd) {
		err = -ENOMEM;
		goto out;
	}

etc.

Also ignoring pte_alloc()'s error code (which will be -ENOMEM anyway) and setting manually is stupid further down so:

- ret = -ENOMEM;
- if (pte_alloc(mm, pmd))
- goto out;
+ err = pte_alloc(mm, pmd);
+ if (err)
+ goto out;

Obviously:

		for (pte = start_pte; pte_idx < batch_size; ++pte, ++pte_idx) {
			int err = insert_page_in_batch_locked(vma, pte,
				addr, pages[curr_page_idx], prot);

->

		for (pte = start_pte; pte_idx < batch_size; ++pte, ++pte_idx) {
			err = ...

Remove horrible ret = err and ret = 0 assignment later.

All this would improve it a lot thanks! :)

Quoted 3 lines

--
2.43.0

Cheers, Lorenzo

#2 lore raw
Quoted 21 lines · Lorenzo Stoakes (ARM)
Quoted 8 lines · Avi Weiss

walk_to_pmd() returns NULL only when p4d_alloc(), pud_alloc(), or pmd_alloc() fails. These are page-table allocation failures, but insert_pages() currently reports them as -EFAULT.

Return -ENOMEM instead, consistent with the subsequent pte_alloc() failure and with the single-page insert_page() path, which reports failure of the same page-table allocation chain as -ENOMEM.

Quoted 5 lines

Address and range validation failures in vm_insert_pages() continue to return -EFAULT. Keep the later -EFAULT return for pte_offset_map_lock(), which is not an allocation failure.

Fixes 8cd3984d81d5 ("mm/memory.c: add vm_insert_pages()")

Hmm :)

This isn't really a fix. Anybody relying on this returning -ENOMEM vs. -EFAULT here is in a state of sin anyway (unless you can point to specific users who are broken).

Quoted 5 lines

Drop the tag.

Quoted 1 line

This is correct, walk

Ugh incomplete thought :)

->

This is correct, the 'walk' can only fail due to failure to allocate.

Anyway I gave you a bunch of stuff to do here, respin once you've addressed all that.

Cheers, Lorenzo

#3 lore raw
In reply to Lorenzo Stoakes (ARM)
Quoted 13 lines · Lorenzo Stoakes (ARM)
Quoted 12 lines · Avi Weiss

walk_to_pmd() returns NULL only when p4d_alloc(), pud_alloc(), or pmd_alloc() fails. These are page-table allocation failures, but insert_pages() currently reports them as -EFAULT.

Return -ENOMEM instead, consistent with the subsequent pte_alloc() failure and with the single-page insert_page() path, which reports failure of the same page-table allocation chain as -ENOMEM.

Address and range validation failures in vm_insert_pages() continue to return -EFAULT. Keep the later -EFAULT return for pte_offset_map_lock(), which is not an allocation failure.

Quoted 7 lines
Quoted 1 line
Fixes 8cd3984d81d5 ("mm/memory.c: add vm_insert_pages()")

Hmm :)

This isn't really a fix. Anybody relying on this returning -ENOMEM vs. -EFAULT here is in a state of sin anyway (unless you can point to specific users who are broken).

Agreed, I am missing the problem here?

Signature · 3 lines
Cheers,

David
#4 lore raw
Quoted 16 lines · David Hildenbrand (Arm)
Quoted 15 lines · Lorenzo Stoakes (ARM)
Quoted 13 lines · Avi Weiss

walk_to_pmd() returns NULL only when p4d_alloc(), pud_alloc(), or pmd_alloc() fails. These are page-table allocation failures, but insert_pages() currently reports them as -EFAULT.

Return -ENOMEM instead, consistent with the subsequent pte_alloc() failure and with the single-page insert_page() path, which reports failure of the same page-table allocation chain as -ENOMEM.

Address and range validation failures in vm_insert_pages() continue to return -EFAULT. Keep the later -EFAULT return for pte_offset_map_lock(), which is not an allocation failure.

Fixes 8cd3984d81d5 ("mm/memory.c: add vm_insert_pages()")
Quoted 7 lines
Quoted 5 lines

Hmm :)

This isn't really a fix. Anybody relying on this returning -ENOMEM vs. -EFAULT here is in a state of sin anyway (unless you can point to specific users who are broken).

Agreed, I am missing the problem here?

Seems just a cleanup, I gave Avi some ideas ;) this function could do with it.

Quoted 5 lines

--
Cheers,

David

Cheers, Lorenzo

maintainer
#5 lore raw
In reply to Lorenzo Stoakes (ARM)
Quoted 10 lines · On Thu, 30 Jul 2026 08:42:49 +0100 "Lorenzo Stoakes (ARM)" <[email protected]>…
	pages_to_write_in_pmd = min_t(unsigned long,
		remaining_pages_total, PTRS_PER_PTE - pte_index(addr));

	/* Allocate the PTE if necessary; takes PMD lock once only. */
	ret = -ENOMEM; <------------------------------ set it again?
	if (pte_alloc(mm, pmd))
		goto out;

The way this function is written is horrible in general, I hate 'preset default return value' as a pattern.

It used to be the preferred way because

	ret = -ENOMEM;
	if (expr)
		goto out;

generated slightly better code than

	if (expr) {
		ret = -ENOMEM;
		goto out;
	}

Whether that is the case with current compilers I don't know.

#6 lore raw
Quoted 19 lines · Andrew Morton
Quoted 10 lines · On Thu, 30 Jul 2026 08:42:49 +0100 "Lorenzo Stoakes (ARM)" <[email protected]>…
	pages_to_write_in_pmd = min_t(unsigned long,
		remaining_pages_total, PTRS_PER_PTE - pte_index(addr));

	/* Allocate the PTE if necessary; takes PMD lock once only. */
	ret = -ENOMEM; <------------------------------ set it again?
	if (pte_alloc(mm, pmd))
		goto out;

The way this function is written is horrible in general, I hate 'preset default return value' as a pattern.

It used to be the preferred way because

	ret = -ENOMEM;
	if (expr)
		goto out;
Quoted 8 lines

generated slightly better code than

	if (expr) {
		ret = -ENOMEM;
		goto out;
	}

Whether that is the case with current compilers I don't know.

Thanks for the background on that!

I'd put money on there being no difference now :) but in any case that kind of micro-optimisation isn't really a concern here even if it was the case.

--
Cheers, Lorenzo