incoming
Subsystems (83)
Andrew Morton
maintainerPATCH 94/181 mm/memory.c: replace vmf->vma with variable vma
In reply to Andrew MortonFrom: Yanfei Xu <[email protected]>
Subject: mm/memory.c: replace vmf->vma with variable vma
The code has declared a vma_struct named vma which is assigned a value of vmf->vma. Thus, use variable vma directly here.
---
mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/memory.c renamed from mm/memory.c~mm-memoryc-replace-vmf-vma-with-variable-vma +1 -1
@@ -3597,7 +3597,7 @@ static vm_fault_t __do_fault(struct vm_f* # flush A, B to clear the writeback */ if (pmd_none (*vmf->pmd) && !vmf->prealloc_pte) { - vmf->prealloc_pte =pte_alloc_one (vmf->vma->vm_mm); + vmf->prealloc_pte =pte_alloc_one (vma->vm_mm);if (!vmf->prealloc_pte)return VM_FAULT_OOM ;smp_wmb ();/* See comment in __pte_alloc() */
_
Andrew Morton
maintainerPATCH 95/181 mm/mmap: rename __vma_unlink_common() to __vma_unlink()
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/mmap: rename __vma_unlink_common() to __vma_unlink()
__vma_unlink_common() and __vma_unlink() are counterparts. Since there is no function named __vma_unlink(), let's rename __vma_unlink_common() to __vma_unlink() to make the code more self-explanatory and easy for audience to understand.
Otherwise we may expect there are several variants of vma_unlink() and __vma_unlink_common() is used by them.
---
mm/mmap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
mm/mmap.c renamed from mm/mmap.c~mm-mmap-rename-__vma_unlink_common-to-__vma_unlink +3 -3
@@ -677,7 +677,7 @@ static void __insert_vm_struct(struct mm mm->map_count++; } -static __always_inline void __vma_unlink_common(struct mm_struct *mm, +static __always_inline void __vma_unlink(struct mm_struct *mm,struct vm_area_struct *vma,struct vm_area_struct *ignore) {
@@ -859,7 +859,7 @@ again:* us to remove next before dropping the locks. */ if (remove_next !=3 ) -__vma_unlink_common (mm, next, next); +__vma_unlink (mm, next, next);else /* * vma is not before next if they've been
@@ -870,7 +870,7 @@ again:* "next" (which is stored in post-swap() * "vma"). */ -__vma_unlink_common (mm, next, vma); +__vma_unlink (mm, next, vma);if (file)__remove_shared_vm_struct (next, file, mapping); }else if (insert) {
_
Andrew Morton
maintainerPATCH 96/181 mm/mmap: leverage vma_rb_erase_ignore() to implement vma_rb_erase()
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/mmap: leverage vma_rb_erase_ignore() to implement vma_rb_erase()
These two functions share the same logic except ignore a different vma.
Let's reuse the code.
---
mm/mmap.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-)
mm/mmap.c renamed from mm/mmap.c~mm-mmap-leverage-vma_rb_erase_ignore-to-implement-vma_rb_erase +7 -9
@@ -474,8 +474,12 @@ static __always_inline void vma_rb_erase {/* * All rb_subtree_gap values must be consistent prior to erase, -* with the possible exception of the "next" vma being erased if -* next->vm_start was reduced. +* with the possible exception of +* +* a. the "next" vma being erased if next->vm_start was reduced in +* __vma_adjust() -> __vma_unlink() +* b. the vma being erased in detach_vmas_to_be_unmapped() -> +* vma_rb_erase() */ validate_mm_rb (root, ignore);
@@ -485,13 +489,7 @@ static __always_inline void vma_rb_erase static __always_inline void vma_rb_erase(struct vm_area_struct *vma,struct rb_root *root) { -/* -* All rb_subtree_gap values must be consistent prior to erase, -* with the possible exception of the vma being erased. -*/ -validate_mm_rb (root, vma); - -__vma_rb_erase (vma, root); +vma_rb_erase_ignore (vma, root, vma); }/*
_
Andrew Morton
maintainerPATCH 97/181 mmap locking API: add mmap_lock_is_contended()
In reply to Andrew MortonFrom: Chinwen Chang <[email protected]>
Subject: mmap locking API: add mmap_lock_is_contended()
Patch series "Try to release mmap_lock temporarily in smaps_rollup", v4.
Recently, we have observed some janky issues caused by unpleasantly long contention on mmap_lock which is held by smaps_rollup when probing large processes. To address the problem, we let smaps_rollup detect if anyone wants to acquire mmap_lock for write attempts. If yes, just release the lock temporarily to ease the contention.
smaps_rollup is a procfs interface which allows users to summarize the process's memory usage without the overhead of seq_* calls. Android uses it to sample the memory usage of various processes to balance its memory pool sizes. If no one wants to take the lock for write requests, smaps_rollup with this patch will behave like the original one.
Although there are on-going mmap_lock optimizations like range-based locks, the lock applied to smaps_rollup would be the coarse one, which is hard to avoid the occurrence of aforementioned issues. So the detection and temporary release for write attempts on mmap_lock in smaps_rollup is still necessary.
This patch (of 3):
Add new API to query if someone wants to acquire mmap_lock for write attempts.
Using this instead of rwsem_is_contended makes it more tolerant of future changes to the lock type.
---
include/linux/mmap_lock.h | 5 +++++ 1 file changed, 5 insertions(+)
include/linux/mmap_lock.h renamed from include/linux/mmap_lock.h~mmap-locking-api-add-mmap_lock_is_contended +5 -0
@@ -87,4 +87,9 @@ static inline void mmap_assert_write_loc VM_BUG_ON_MM(!rwsem_is_locked(&mm->mmap_lock), mm); } +static inline int mmap_lock_is_contended (struct mm_struct *mm) +{ +return rwsem_is_contended (&mm->mmap_lock); +} +#endif /* _LINUX_MMAP_LOCK_H */
_
Andrew Morton
maintainerPATCH 98/181 mm: smaps*: extend smap_gather_stats to support specified beginning
In reply to Andrew MortonFrom: Chinwen Chang <[email protected]>
Subject: mm: smaps*: extend smap_gather_stats to support specified beginning
Extend smap_gather_stats to support indicated beginning address at which it should start gathering. To achieve the goal, we add a new parameter @start assigned by the caller and try to refactor it for simplicity.
If @start is 0, it will use the range of @vma for gathering.
---
fs/proc/task_mmu.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-)
fs/proc/task_mmu.c renamed from fs/proc/task_mmu.c~mm-smaps-extend-smap_gather_stats-to-support-specified-beginning +22 -8
@@ -721,9 +721,21 @@ static const struct mm_walk_ops smaps_sh .pte_hole = smaps_pte_hole, }; +/* +* Gather mem stats from @vma with the indicated beginning +* address @start, and keep them in @mss. +* +* Use vm_start of @vma as the beginning address if @start is 0. +*/ static void smap_gather_stats (struct vm_area_struct *vma, -struct mem_size_stats *mss) +struct mem_size_stats *mss,unsigned long start) { +const struct mm_walk_ops *ops = &smaps_walk_ops; + +/* Invalid start */ +if (start >= vma->vm_end) +return ; +#ifdef CONFIG_SHMEM /* In case of smaps_rollup, reset the value from previous vma */ mss->check_shmem_swap = false;
@@ -740,18 +752,20 @@ static void smap_gather_stats(struct vm_*/ unsigned long shmem_swapped =shmem_swap_usage (vma); -if (!shmem_swapped || (vma->vm_flags &VM_SHARED ) || - !(vma->vm_flags &VM_WRITE )) { +if (!start && (!shmem_swapped || (vma->vm_flags &VM_SHARED ) || + !(vma->vm_flags &VM_WRITE ))) { mss->swap += shmem_swapped; }else { mss->check_shmem_swap = true; -walk_page_vma (vma, &smaps_shmem_walk_ops, mss); -return ; + ops = &smaps_shmem_walk_ops; } }#endif /* mmap_lock is held in m_start */ -walk_page_vma (vma, &smaps_walk_ops, mss); +if (!start) +walk_page_vma (vma, ops, mss); +else +walk_page_range (vma->vm_mm, start, vma->vm_end, ops, mss); }#define SEQ_PUT_DEC (str, val) \
@@ -803,7 +817,7 @@ static int show_smap(struct seq_file *m,memset (&mss,0 ,sizeof (mss)); -smap_gather_stats (vma, &mss); +smap_gather_stats (vma, &mss,0 );show_map_vma (m, vma);
@@ -852,7 +866,7 @@ static int show_smaps_rollup(struct seq_hold_task_mempolicy (priv);for (vma = priv->mm->mmap; vma; vma = vma->vm_next) { -smap_gather_stats (vma, &mss); +smap_gather_stats (vma, &mss,0 ); last_vma_end = vma->vm_end; }
_
Andrew Morton
maintainerPATCH 99/181 mm: proc: smaps_rollup: do not stall write attempts on mmap_lock
In reply to Andrew MortonFrom: Chinwen Chang <[email protected]>
Subject: mm: proc: smaps_rollup: do not stall write attempts on mmap_lock
smaps_rollup will try to grab mmap_lock and go through the whole vma list until it finishes the iterating. When encountering large processes, the mmap_lock will be held for a longer time, which may block other write requests like mmap and munmap from progressing smoothly.
There are upcoming mmap_lock optimizations like range-based locks, but the lock applied to smaps_rollup would be the coarse type, which doesn't avoid the occurrence of unpleasant contention.
To solve aforementioned issue, we add a check which detects whether anyone wants to grab mmap_lock for write attempts.
---
fs/proc/task_mmu.c | 66 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 65 insertions(+), 1 deletion(-)
fs/proc/task_mmu.c renamed from fs/proc/task_mmu.c~mm-proc-smaps_rollup-do-not-stall-write-attempts-on-mmap_lock +65 -1
@@ -865,9 +865,73 @@ static int show_smaps_rollup(struct seq_hold_task_mempolicy (priv); -for (vma = priv->mm->mmap; vma; vma = vma->vm_next) { +for (vma = priv->mm->mmap; vma;) {smap_gather_stats (vma, &mss,0 ); last_vma_end = vma->vm_end; + +/* +* Release mmap_lock temporarily if someone wants to +* access it for write request. +*/ +if (mmap_lock_is_contended (mm)) { +mmap_read_unlock (mm); + ret =mmap_read_lock_killable (mm); +if (ret) { +release_task_mempolicy (priv); + goto out_put_mm; + } + +/* +* After dropping the lock, there are four cases to +* consider. See the following example for explanation. +* +* +------+------+-----------+ +* | VMA1 | VMA2 | VMA3 | +* +------+------+-----------+ +* | | | | +* 4k 8k 16k 400k +* +* Suppose we drop the lock after reading VMA2 due to +* contention, then we get: +* +* last_vma_end = 16k +* +* 1) VMA2 is freed, but VMA3 exists: +* +* find_vma(mm, 16k - 1) will return VMA3. +* In this case, just continue from VMA3. +* +* 2) VMA2 still exists: +* +* find_vma(mm, 16k - 1) will return VMA2. +* Iterate the loop like the original one. +* +* 3) No more VMAs can be found: +* +* find_vma(mm, 16k - 1) will return NULL. +* No more things to do, just break. +* +* 4) (last_vma_end - 1) is the middle of a vma (VMA'): +* +* find_vma(mm, 16k - 1) will return VMA' whose range +* contains last_vma_end. +* Iterate VMA' from last_vma_end. +*/ + vma =find_vma (mm, last_vma_end -1 ); +/* Case 3 above */ +if (!vma) +break ; + +/* Case 1 above */ +if (vma->vm_start >= last_vma_end) +continue ; + +/* Case 4 above */ +if (vma->vm_end > last_vma_end) +smap_gather_stats (vma, &mss, last_vma_end); + } +/* Case 2 above */ + vma = vma->vm_next; } show_vma_header_prefix(m, priv->mm->mmap->vm_start,
_
Andrew Morton
maintainerPATCH 100/181 mm: move PageDoubleMap bit
In reply to Andrew MortonFrom: "Matthew Wilcox (Oracle)" <[email protected]>
Subject: mm: move PageDoubleMap bit
Patch series "Fix PageDoubleMap".
This is a purely theoretical problem for now as none of the filesystems which use PG_private_2 (ie PG_fscache) are being converted at this time, but it's confusing to leave it like this.
This patch (of 2):
PG_private_2 is defined as being PF_ANY (applicable to tail pages as well as regular & head pages). That means that the first tail page of a double-map page will appear to have Private2 set. Use the Workingset bit instead which is defined as PF_HEAD so any attempt to access the Workingset bit on a tail page will redirect to the head page's Workingset bit.
---
include/linux/page-flags.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/page-flags.h renamed from include/linux/page-flags.h~mm-move-pagedoublemap-bit +1 -1
@@ -167,7 +167,7 @@ enum pageflags { PG_slob_free = PG_private,/* Compound pages. Stored in first tail page's flags */ - PG_double_map = PG_private_2, + PG_double_map = PG_workingset,/* non-lru isolated movable page */ PG_isolated = PG_reclaim,
_
Andrew Morton
maintainerPATCH 101/181 mm: simplify PageDoubleMap with PF_SECOND policy
In reply to Andrew MortonFrom: "Matthew Wilcox (Oracle)" <[email protected]>
Subject: mm: simplify PageDoubleMap with PF_SECOND policy
Introduce the new page policy of PF_SECOND which lets us use the normal pageflags generation machinery to create the various DoubleMap manipulation functions.
---
include/linux/page-flags.h | 40 ++++++++--------------------------- 1 file changed, 10 insertions(+), 30 deletions(-)
include/linux/page-flags.h renamed from include/linux/page-flags.h~mm-simplify-pagedoublemap-with-pf_second-policy +10 -30
@@ -235,6 +235,9 @@ static inline void page_init_poison(stru* * PF_NO_COMPOUND: * the page flag is not relevant for compound pages. +* +* PF_SECOND: +* the page flag is stored in the first tail page. */ #define PF_POISONED_CHECK (page) ({ \ VM_BUG_ON_PGFLAGS(PagePoisoned(page), page); \
@@ -250,6 +253,9 @@ static inline void page_init_poison(stru#define PF_NO_COMPOUND (page, enforce) ({ \ VM_BUG_ON_PGFLAGS(enforce && PageCompound(page), page); \ PF_POISONED_CHECK(page); }) +#define PF_SECOND (page, enforce) ({ \ + VM_BUG_ON_PGFLAGS(!PageHead(page), page); \ + PF_POISONED_CHECK(&page[1]); })/* * Macros to create function definitions for page flags
@@ -688,42 +694,15 @@ static inline int PageTransTail(struct p* * See also __split_huge_pmd_locked() and page_remove_anon_compound_rmap(). */ -static inline int PageDoubleMap (struct page *page) -{ -return PageHead (page) &&test_bit (PG_double_map, &page[1 ].flags); -} - -static inline void SetPageDoubleMap (struct page *page) -{ -VM_BUG_ON_PAGE (!PageHead (page), page); -set_bit (PG_double_map, &page[1 ].flags); -} - -static inline void ClearPageDoubleMap (struct page *page) -{ -VM_BUG_ON_PAGE (!PageHead (page), page); -clear_bit (PG_double_map, &page[1 ].flags); -} -static inline int TestSetPageDoubleMap (struct page *page) -{ -VM_BUG_ON_PAGE (!PageHead (page), page); -return test_and_set_bit (PG_double_map, &page[1 ].flags); -} - -static inline int TestClearPageDoubleMap (struct page *page) -{ -VM_BUG_ON_PAGE (!PageHead (page), page); -return test_and_clear_bit (PG_double_map, &page[1 ].flags); -} - +PAGEFLAG(DoubleMap, double_map, PF_SECOND) + TESTSCFLAG(DoubleMap, double_map, PF_SECOND)#else TESTPAGEFLAG_FALSE(TransHuge) TESTPAGEFLAG_FALSE(TransCompound) TESTPAGEFLAG_FALSE(TransCompoundMap) TESTPAGEFLAG_FALSE(TransTail) PAGEFLAG_FALSE(DoubleMap) - TESTSETFLAG_FALSE(DoubleMap) - TESTCLEARFLAG_FALSE(DoubleMap) + TESTSCFLAG_FALSE(DoubleMap)#endif /*
@@ -888,6 +867,7 @@ static inline int page_has_private(struc#undef PF_ONLY_HEAD#undef PF_NO_TAIL#undef PF_NO_COMPOUND +#undef PF_SECOND#endif /* !__GENERATING_BOUNDS_H */ #endif /* PAGE_FLAGS_H */
_
Andrew Morton
maintainerPATCH 102/181 mm/mmap: leave adjust_next as virtual address instead of page frame number
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/mmap: leave adjust_next as virtual address instead of page frame number
Instead of converting adjust_next between bytes and pages number, let's just store the virtual address into adjust_next.
Also, this patch fixes one typo in the comment of vma_adjust_trans_huge().
---
mm/huge_memory.c | 4 ++-- mm/mmap.c | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-)
mm/huge_memory.c renamed from mm/huge_memory.c~mm-mmap-leave-adjust_next-as-virtual-address-instead-of-page-frame-number +2 -2
@@ -2306,13 +2306,13 @@ void vma_adjust_trans_huge(struct vm_are/* * If we're also updating the vma->vm_next->vm_start, if the new -* vm_next->vm_start isn't page aligned and it could previously +* vm_next->vm_start isn't hpage aligned and it could previously * contain an hugepage: check if we need to split an huge pmd. */ if (adjust_next >0 ) {struct vm_area_struct *next = vma->vm_next;unsigned long nstart = next->vm_start; - nstart += adjust_next <<PAGE_SHIFT ; + nstart += adjust_next;if (nstart & ~HPAGE_PMD_MASK && (nstart &HPAGE_PMD_MASK ) >= next->vm_start && (nstart &HPAGE_PMD_MASK ) +HPAGE_PMD_SIZE <= next->vm_end)
mm/mmap.c renamed from mm/mmap.c~mm-mmap-leave-adjust_next-as-virtual-address-instead-of-page-frame-number +4 -4
@@ -758,7 +758,7 @@ int __vma_adjust(struct vm_area_struct ** vma expands, overlapping part of the next: * mprotect case 5 shifting the boundary up. */ - adjust_next = (end - next->vm_start) >>PAGE_SHIFT ; + adjust_next = (end - next->vm_start); exporter = next; importer = vma;VM_WARN_ON (expand != importer);
@@ -768,7 +768,7 @@ int __vma_adjust(struct vm_area_struct ** split_vma inserting another: so it must be * mprotect case 4 shifting the boundary down. */ - adjust_next = -((vma->vm_end - end) >>PAGE_SHIFT ); + adjust_next = -(vma->vm_end - end); exporter = vma; importer = next;VM_WARN_ON (expand != importer);
@@ -840,8 +840,8 @@ again: } vma->vm_pgoff = pgoff;if (adjust_next) { - next->vm_start += adjust_next <<PAGE_SHIFT ; - next->vm_pgoff += adjust_next; + next->vm_start += adjust_next; + next->vm_pgoff += adjust_next >>PAGE_SHIFT ; }if (root) {
_
Andrew Morton
maintainerPATCH 103/181 mm/memory.c: fix spello of "function"
In reply to Andrew MortonFrom: Randy Dunlap <[email protected]>
Subject: mm/memory.c: fix spello of "function"
Fix typo/spello of "function".
---
mm/memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/memory.c renamed from mm/memory.c~mm-memoryc-fix-spello-of-function +1 -1
@@ -3764,7 +3764,7 @@ static vm_fault_t do_set_pmd(struct vm_f/** * alloc_set_pte - setup new PTE entry for given page and add reverse page -* mapping. If needed, the fucntion allocates page table or use pre-allocated. +* mapping. If needed, the function allocates page table or use pre-allocated. * * @vmf: fault environment * @page: page to map
_
Andrew Morton
maintainerPATCH 104/181 mm/mmap: not necessary to check mapping separately
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/mmap: not necessary to check mapping separately
*root* with type of struct rb_root_cached is an element of *mapping* with type of struct address_space. This implies when we have a valid *root* it must be a part of valid *mapping*.
So we can merge these two checks together to make the code more easy to read and to save some cpu cycles.
---
mm/mmap.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
mm/mmap.c renamed from mm/mmap.c~mm-mmap-not-necessary-to-check-mapping-separately +1 -2
@@ -895,10 +895,9 @@ again:anon_vma_interval_tree_post_update_vma (next);anon_vma_unlock_write (anon_vma); } -if (mapping) -i_mmap_unlock_write (mapping);if (root) { +i_mmap_unlock_write (mapping);uprobe_mmap (vma);if (adjust_next)
_
Andrew Morton
maintainerPATCH 105/181 mm/mmap: check on file instead of the rb_root_cached of its address_space
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/mmap: check on file instead of the rb_root_cached of its address_space
In __vma_adjust(), we do the check on *root* to decide whether to adjust the address_space. It seems to be more meaningful to do the check on *file* itself. This means we are adjusting some data because it is a file backed vma.
Since we seem to assume the address_space is valid if it is a file backed vma, let's just replace *root* with *file* here.
---
mm/mmap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
mm/mmap.c renamed from mm/mmap.c~mm-mmap-check-on-file-instead-of-the-rb_root_cached-of-its-address_space +3 -3
@@ -823,7 +823,7 @@ again:anon_vma_interval_tree_pre_update_vma (next); } -if (root) { +if (file) {flush_dcache_mmap_lock (mapping);vma_interval_tree_remove (vma, root);if (adjust_next)
@@ -844,7 +844,7 @@ again: next->vm_pgoff += adjust_next >>PAGE_SHIFT ; } -if (root) { +if (file) {if (adjust_next)vma_interval_tree_insert (next, root);vma_interval_tree_insert (vma, root);
@@ -896,7 +896,7 @@ again:anon_vma_unlock_write (anon_vma); } -if (root) { +if (file) {i_mmap_unlock_write (mapping);uprobe_mmap (vma);
_
Andrew Morton
maintainerPATCH 106/181 mm: use helper function mapping_allow_writable()
In reply to Andrew MortonFrom: Miaohe Lin <[email protected]>
Subject: mm: use helper function mapping_allow_writable()
Commit 4bb5f5d9395b ("mm: allow drivers to prevent new writable mappings") changed i_mmap_writable from unsigned int to atomic_t and add the helper function mapping_allow_writable() to atomic_inc i_mmap_writable. But it forgot to use this helper function in dup_mmap() and __vma_link_file().
---
kernel/fork.c | 2 +- mm/mmap.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
kernel/fork.c renamed from kernel/fork.c~mm-use-helper-function-mapping_allow_writable +1 -1
@@ -559,7 +559,7 @@ static __latent_entropy int dup_mmap(stratomic_dec (&inode->i_writecount);i_mmap_lock_write (mapping);if (tmp->vm_flags &VM_SHARED ) -atomic_inc (&mapping->i_mmap_writable); +mapping_allow_writable (mapping);flush_dcache_mmap_lock (mapping);/* insert tmp into the share list, just after mpnt */ vma_interval_tree_insert_after(tmp, mpnt,
mm/mmap.c renamed from mm/mmap.c~mm-use-helper-function-mapping_allow_writable +1 -1
@@ -621,7 +621,7 @@ static void __vma_link_file(struct vm_arif (vma->vm_flags &VM_DENYWRITE )atomic_dec (&file_inode (file)->i_writecount);if (vma->vm_flags &VM_SHARED ) -atomic_inc (&mapping->i_mmap_writable); +mapping_allow_writable (mapping);flush_dcache_mmap_lock (mapping);vma_interval_tree_insert (vma, &mapping->i_mmap);
_
Andrew Morton
maintainerPATCH 107/181 mm/mmap.c: use helper function allow_write_access() in __remove_shared_vm_struct()
In reply to Andrew MortonFrom: Miaohe Lin <[email protected]>
Subject: mm/mmap.c: use helper function allow_write_access() in __remove_shared_vm_struct()
In commit 1da177e4c3f4 ("Linux-2.6.12-rc2"), the helper allow_write_access came with the atomic_inc operation of the i_writecount field in the func __remove_shared_vm_struct(). But it forgot to use this helper function.
---
mm/mmap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/mmap.c renamed from mm/mmap.c~mm-mmap-use-helper-function-allow_write_access-in-__remove_shared_vm_struct +1 -1
@@ -143,7 +143,7 @@ static void __remove_shared_vm_struct(st struct file *file, struct address_space *mapping) {if (vma->vm_flags &VM_DENYWRITE ) -atomic_inc (&file_inode (file)->i_writecount); +allow_write_access (file);if (vma->vm_flags &VM_SHARED )mapping_unmap_writable (mapping);
_
Andrew Morton
maintainerPATCH 108/181 mm/mmap.c: replace do_brk with do_brk_flags in comment of insert_vm_struct()
In reply to Andrew MortonFrom: Liao Pingfang <[email protected]>
Subject: mm/mmap.c: replace do_brk with do_brk_flags in comment of insert_vm_struct()
Replace do_brk with do_brk_flags in comment of insert_vm_struct(), since do_brk was removed in following commit.
---
mm/mmap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/mmap.c renamed from mm/mmap.c~mm-mmapc-replace-do_brk-with-do_brk_flags-in-comment-of-insert_vm_struct +1 -1
@@ -3233,7 +3233,7 @@ int insert_vm_struct(struct mm_struct *m* By setting it to reflect the virtual start address of the * vma, merges and splits can happen in a seamless way, just * using the existing file pgoff checks and manipulations. -* Similarly in do_mmap and in do_brk. +* Similarly in do_mmap and in do_brk_flags. */ if (vma_is_anonymous (vma)) {BUG_ON (vma->anon_vma);
_
Andrew Morton
maintainerPATCH 109/181 mm: remove src/dst mm parameter in copy_page_range()
In reply to Andrew MortonFrom: Peter Xu <[email protected]>
Subject: mm: remove src/dst mm parameter in copy_page_range()
Both of the mm pointers are not needed after commit 7a4830c380f3 ("mm/fork: Pass new vma pointer into copy_page_range()").
Jason Gunthorpe also reported that the ordering of copy_page_range() is odd. Since working at it, reorder the parameters to be logical, by (1) always put the dst_* fields to be before src_* fields, and (2) keep the same type of parameters together.
---
include/linux/mm.h | 4 - kernel/fork.c | 2 mm/memory.c | 141 ++++++++++++++++++++++--------------------- 3 files changed, 76 insertions(+), 71 deletions(-)
include/linux/mm.h renamed from include/linux/mm.h~mm-remove-src-dst-mm-parameter-in-copy_page_range +2 -2
@@ -1653,8 +1653,8 @@ struct mmu_notifier_range;void free_pgd_range (struct mmu_gather *tlb,unsigned long addr,unsigned long end,unsigned long floor,unsigned long ceiling); -int copy_page_range (struct mm_struct *dst,struct mm_struct *src, -struct vm_area_struct *vma,struct vm_area_struct *new); +int +copy_page_range (struct vm_area_struct *dst_vma,struct vm_area_struct *src_vma);int follow_pte_pmd (struct mm_struct *mm,unsigned long address,struct mmu_notifier_range *range,pte_t **ptepp,pmd_t **pmdpp,spinlock_t **ptlp);
kernel/fork.c renamed from kernel/fork.c~mm-remove-src-dst-mm-parameter-in-copy_page_range +1 -1
@@ -590,7 +590,7 @@ static __latent_entropy int dup_mmap(str mm->map_count++;if (!(tmp->vm_flags &VM_WIPEONFORK )) - retval =copy_page_range (mm, oldmm, mpnt, tmp); + retval =copy_page_range (tmp, mpnt);if (tmp->vm_ops && tmp->vm_ops->open) tmp->vm_ops->open (tmp);
mm/memory.c renamed from mm/memory.c~mm-remove-src-dst-mm-parameter-in-copy_page_range +73 -68
291 lines not on this page · raw message
_
Andrew Morton
maintainerPATCH 110/181 include/linux/huge_mm.h: remove mincore_huge_pmd declaration
In reply to Andrew MortonFrom: yuleixzhang <[email protected]>
Subject: include/linux/huge_mm.h: remove mincore_huge_pmd declaration
As mincore_huge_pmd() was dropped, remove the declaration from the header file.
---
include/linux/huge_mm.h | 3 --- 1 file changed, 3 deletions(-)
include/linux/huge_mm.h renamed from include/linux/huge_mm.h~mm-cleanup-mincore_huge_pmd +0 -3
@@ -38,9 +38,6 @@ extern int zap_huge_pmd(struct mmu_gatheextern int zap_huge_pud (struct mmu_gather *tlb,struct vm_area_struct *vma,pud_t *pud,unsigned long addr); -extern int mincore_huge_pmd (struct vm_area_struct *vma,pmd_t *pmd, -unsigned long addr,unsigned long end, -unsigned char *vec);extern bool move_huge_pmd (struct vm_area_struct *vma,unsigned long old_addr,unsigned long new_addr,pmd_t *old_pmd,pmd_t *new_pmd);
_
Andrew Morton
maintainerPATCH 111/181 tools/testing/selftests/vm/hmm-tests.c: use the new SKIP() macro
In reply to Andrew MortonFrom: Ralph Campbell <[email protected]>
Subject: tools/testing/selftests/vm/hmm-tests.c: use the new SKIP() macro
Some tests might not be able to be run if resources like huge pages are not available. Mark these tests as skipped instead of simply passing.
---
tools/testing/selftests/vm/hmm-tests.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/testing/selftests/vm/hmm-tests.c renamed from tools/testing/selftests/vm/hmm-tests.c~mm-test-use-the-new-skip-macro +2 -2
@@ -680,7 +680,7 @@ TEST_F(hmm, anon_write_hugetlbfs) n =gethugepagesizes (pagesizes,4 );if (n <=0 ) -return ; +SKIP (return,"Huge page size could not be determined" );for (idx =0 ; --n >0 ; ) {if (pagesizes[n] < pagesizes[idx]) idx = n;
@@ -694,7 +694,7 @@ TEST_F(hmm, anon_write_hugetlbfs) buffer->ptr =get_hugepage_region (size,GHR_STRICT );if (buffer->ptr ==NULL ) {free (buffer); -return ; +SKIP (return,"Huge page could not be allocated" ); } buffer->fd =-1 ;
_
Andrew Morton
maintainerPATCH 112/181 lib/test_hmm.c: remove unused dmirror_zero_page
In reply to Andrew MortonFrom: Ralph Campbell <[email protected]>
Subject: lib/test_hmm.c: remove unused dmirror_zero_page
The variable dmirror_zero_page is unused in the HMM self test driver which was probably intended to demonstrate how a driver could use migrate_vma_setup() to share a single read-only device private zero page similar to how the CPU does. However, this isn't needed for the self tests so remove it.
---
lib/test_hmm.c | 14 -------------- 1 file changed, 14 deletions(-)
lib/test_hmm.c renamed from lib/test_hmm.c~hmm-test-remove-unused-dmirror_zero_page +0 -14
@@ -36,7 +36,6 @@static const struct dev_pagemap_ops dmirror_devmem_ops;static const struct mmu_interval_notifier_ops dmirror_min_ops;static dev_t dmirror_dev; -static struct page *dmirror_zero_page;struct dmirror_device ;
@@ -1127,17 +1126,6 @@ static int __init hmm_dmirror_init(void) goto err_chrdev; } -/* -* Allocate a zero page to simulate a reserved page of device private -* memory which is always zero. The zero_pfn page isn't used just to -* make the code here simpler (i.e., we need a struct page for it). -*/ - dmirror_zero_page =alloc_page (GFP_HIGHUSER | __GFP_ZERO); -if (!dmirror_zero_page) { - ret = -ENOMEM ; - goto err_chrdev; - } -pr_info ("HMM test module loaded. This is only for testing HMM.\n" );return 0 ;
@@ -1153,8 +1141,6 @@ static void __exit hmm_dmirror_exit(void {int id; -if (dmirror_zero_page) -__free_page (dmirror_zero_page);for (id =0 ; id <DMIRROR_NDEVICES ; id++)dmirror_device_remove (dmirror_devices + id);unregister_chrdev_region (dmirror_dev,DMIRROR_NDEVICES );
_
Andrew Morton
maintainerPATCH 113/181 mm/dmapool.c: replace open-coded list_for_each_entry_safe()
In reply to Andrew MortonFrom: Andy Shevchenko <[email protected]>
Subject: mm/dmapool.c: replace open-coded list_for_each_entry_safe()
There is a place in the code where open-coded version of list_for_each_entry_safe() is used. Replace that with the standard macro.
---
mm/dmapool.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)
mm/dmapool.c renamed from mm/dmapool.c~mm-dmapoolc-replace-open-coded-list_for_each_entry_safe +2 -4
@@ -266,6 +266,7 @@ static void pool_free_page(struct dma_po*/ void dma_pool_destroy (struct dma_pool *pool) { +struct dma_page *page, *tmp;bool empty = false;if (unlikely (!pool))
@@ -281,10 +282,7 @@ void dma_pool_destroy(struct dma_pool *pdevice_remove_file (pool->dev, &dev_attr_pools);mutex_unlock (&pools_reg_lock); -while (!list_empty (&pool->page_list)) { -struct dma_page *page; - page =list_entry (pool->page_list.next, - struct dma_page, page_list); +list_for_each_entry_safe (page, tmp, &pool->page_list, page_list) {if (is_page_busy (page)) {if (pool->dev) dev_err(pool->dev,
_
Andrew Morton
maintainerPATCH 114/181 mm/dmapool.c: replace hard coded function name with __func__
In reply to Andrew MortonFrom: Andy Shevchenko <[email protected]>
Subject: mm/dmapool.c: replace hard coded function name with __func__
No need to hard code function name when __func__ can be used.
While here, replace specifiers for special types like dma_addr_t.
---
mm/dmapool.c | 40 ++++++++++++++++++---------------------- 1 file changed, 18 insertions(+), 22 deletions(-)
mm/dmapool.c renamed from mm/dmapool.c~mm-dmapoolc-replace-hard-coded-function-name-with-__func__ +18 -22
@@ -285,11 +285,10 @@ void dma_pool_destroy(struct dma_pool *plist_for_each_entry_safe (page, tmp, &pool->page_list, page_list) {if (is_page_busy (page)) {if (pool->dev) -dev_err (pool->dev, -"dma_pool_destroy %s, %p busy\n" , +dev_err (pool->dev,"%s %s, %p busy\n" , __func__, pool->name, page->vaddr);else -pr_err ("dma_pool_destroy %s, %p busy\n" , +pr_err ("%s %s, %p busy\n" , __func__, pool->name, page->vaddr);/* leak the still-in-use consistent memory */ list_del (&page->page_list);
@@ -353,12 +352,11 @@ void *dma_pool_alloc(struct dma_pool *poif (data[i] ==POOL_POISON_FREED )continue ;if (pool->dev) -dev_err (pool->dev, -"dma_pool_alloc %s, %p (corrupted)\n" , - pool->name, retval); +dev_err (pool->dev,"%s %s, %p (corrupted)\n" , + __func__, pool->name, retval);else -pr_err ("dma_pool_alloc %s, %p (corrupted)\n" , - pool->name, retval); +pr_err ("%s %s, %p (corrupted)\n" , + __func__, pool->name, retval);/* * Dump the first 4 bytes even if they are not
@@ -414,12 +412,11 @@ void dma_pool_free(struct dma_pool *poolif (!page) {spin_unlock_irqrestore (&pool->lock, flags);if (pool->dev) -dev_err (pool->dev, -"dma_pool_free %s, %p/%lx (bad dma)\n" , - pool->name, vaddr, (unsigned long )dma); +dev_err (pool->dev,"%s %s, %p/%pad (bad dma)\n" , + __func__, pool->name, vaddr, &dma);else -pr_err ("dma_pool_free %s, %p/%lx (bad dma)\n" , - pool->name, vaddr, (unsigned long )dma); +pr_err ("%s %s, %p/%pad (bad dma)\n" , + __func__, pool->name, vaddr, &dma);return ; }
@@ -430,12 +427,11 @@ void dma_pool_free(struct dma_pool *poolif ((dma - page->dma) != offset) {spin_unlock_irqrestore (&pool->lock, flags);if (pool->dev) -dev_err (pool->dev, -"dma_pool_free %s, %p (bad vaddr)/%pad\n" , - pool->name, vaddr, &dma); +dev_err (pool->dev,"%s %s, %p (bad vaddr)/%pad\n" , + __func__, pool->name, vaddr, &dma);else -pr_err ("dma_pool_free %s, %p (bad vaddr)/%pad\n" , - pool->name, vaddr, &dma); +pr_err ("%s %s, %p (bad vaddr)/%pad\n" , + __func__, pool->name, vaddr, &dma);return ; } {
@@ -447,11 +443,11 @@ void dma_pool_free(struct dma_pool *pool }spin_unlock_irqrestore (&pool->lock, flags);if (pool->dev) -dev_err (pool->dev,"dma_pool_free %s, dma %pad already free\n" , - pool->name, &dma); +dev_err (pool->dev,"%s %s, dma %pad already free\n" , + __func__, pool->name, &dma);else -pr_err ("dma_pool_free %s, dma %pad already free\n" , - pool->name, &dma); +pr_err ("%s %s, dma %pad already free\n" , + __func__, pool->name, &dma);return ; } }
_
Andrew Morton
maintainerPATCH 115/181 mm/memory-failure: do pgoff calculation before for_each_process()
In reply to Andrew MortonFrom: Xianting Tian <[email protected]>
Subject: mm/memory-failure: do pgoff calculation before for_each_process()
There is no need to calculate pgoff in each loop of for_each_process(), so move it to the place before for_each_process(), which can save some CPU cycles.
---
mm/memory-failure.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mm/memory-failure.c renamed from mm/memory-failure.c~mm-memory-failure-do-pgoff-calculation-before-for_each_process +2 -1
@@ -484,11 +484,12 @@ static void collect_procs_file(struct pastruct vm_area_struct *vma;struct task_struct *tsk;struct address_space *mapping = page->mapping; +pgoff_t pgoff;i_mmap_lock_read (mapping);read_lock (&tasklist_lock); + pgoff =page_to_pgoff (page);for_each_process (tsk) { -pgoff_t pgoff =page_to_pgoff (page);struct task_struct *t =task_early_kill (tsk, force_early);if (!t)
_
Andrew Morton
maintainerPATCH 116/181 mm/memory-failure.c: remove unused macro `writeback'
In reply to Andrew MortonFrom: Alex Shi <[email protected]>
Subject: mm/memory-failure.c: remove unused macro `writeback'
Unlike others we don't use the marco writeback. so let's remove it to tame gcc warning:
mm/memory-failure.c:827: warning: macro "writeback" is not used [-Wunused-macros]
---
mm/memory-failure.c | 2 -- 1 file changed, 2 deletions(-)
mm/memory-failure.c renamed from mm/memory-failure.c~mm-remove-unused-marco-writeback +0 -2
@@ -825,7 +825,6 @@ static int me_huge_page(struct page *p,#define sc ((1UL << PG_swapcache) | (1UL << PG_swapbacked))#define unevict (1UL << PG_unevictable)#define mlock (1UL << PG_mlocked) -#define writeback (1UL << PG_writeback)#define lru (1UL << PG_lru)#define head (1UL << PG_head)#define slab (1UL << PG_slab)
@@ -874,7 +873,6 @@ static struct page_state {#undef sc#undef unevict#undef mlock -#undef writeback#undef lru#undef head#undef slab
_
Andrew Morton
maintainerPATCH 117/181 mm/vmalloc.c: update the comment in __vmalloc_area_node()
In reply to Andrew MortonFrom: Hui Su <[email protected]>
Subject: mm/vmalloc.c: update the comment in __vmalloc_area_node()
Since c67dc624757 ("mm/vmalloc: do not call kmemleak_free() on not yet accounted memory"), the __vunmap() have been changed to __vfree(), so update the confusing comment().
---
mm/vmalloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmalloc.c renamed from mm/vmalloc.c~mm-vmallocc-update-the-comment-in-__vmalloc_area_node +1 -1
@@ -2447,7 +2447,7 @@ static void *__vmalloc_area_node(struct page =alloc_pages_node (node, alloc_mask|highmem_mask,0 );if (unlikely (!page)) { -/* Successfully allocated i pages, free them in __vunmap() */ +/* Successfully allocated i pages, free them in __vfree() */ area->nr_pages = i;atomic_long_add (area->nr_pages, &nr_vmalloc_pages); goto fail;
_
Andrew Morton
maintainerPATCH 118/181 mm/vmalloc.c: fix the comment of find_vm_area
In reply to Andrew MortonFrom: Hui Su <[email protected]>
Subject: mm/vmalloc.c: fix the comment of find_vm_area
Fix the comment of find_vm_area() and get_vm_area()
---
mm/vmalloc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/vmalloc.c renamed from mm/vmalloc.c~mm-vmallocc-fix-the-comment-of-find_vm_area +2 -2
@@ -2133,7 +2133,7 @@ struct vm_struct *get_vm_area_caller(uns* It is up to the caller to do all required locking to keep the returned * pointer valid. * -* Return: pointer to the found area or %NULL on faulure +* Return: the area descriptor on success or %NULL on failure. */ struct vm_struct *find_vm_area (const void *addr) {
@@ -2154,7 +2154,7 @@ struct vm_struct *find_vm_area(const voi* This function returns the found VM area, but using it is NOT safe * on SMP machines, except for its size or flags. * -* Return: pointer to the found area or %NULL on faulure +* Return: the area descriptor on success or %NULL on failure. */ struct vm_struct *remove_vm_area (const void *addr) {
_
Andrew Morton
maintainerPATCH 119/181 docs/vm: fix 'mm_count' vs 'mm_users' counter confusion
In reply to Andrew MortonFrom: Alexander Gordeev <[email protected]>
Subject: docs/vm: fix 'mm_count' vs 'mm_users' counter confusion
In the context of the anonymous address space lifespan description the 'mm_users' reference counter is confused with 'mm_count'. I.e a "zombie" mm gets released when "mm_count" becomes zero, not "mm_users".
---
Documentation/vm/active_mm.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/vm/active_mm.rst renamed from Documentation/vm/active_mm.rst~docs-vm-fix-mm_count-vs-mm_users-counter-confusion +1 -1
@@ -64,7 +64,7 @@ Active MM actually get cases where you have a address space that is _only_ used by lazy users. That is often a short-lived state, because once that thread gets scheduled away in favour of a real thread, the "zombie" mm gets - released because "mm_users" becomes zero. + released because "mm_count" becomes zero. Also, a new rule is that _nobody_ ever has "init_mm" as a real MM any more. "init_mm" should be considered just a "lazy context when no other
_
Andrew Morton
maintainerPATCH 120/181 kasan/kunit: add KUnit Struct to Current Task
In reply to Andrew MortonFrom: Patricia Alfonso <[email protected]>
Subject: kasan/kunit: add KUnit Struct to Current Task
Patch series "KASAN-KUnit Integration", v14.
This patchset contains everything needed to integrate KASAN and KUnit.
KUnit will be able to:
(1) Fail tests when an unexpected KASAN error occurs
(2) Pass tests when an expected KASAN error occurs
Convert KASAN tests to KUnit with the exception of copy_user_test because KUnit is unable to test those.
Add documentation on how to run the KASAN tests with KUnit and what to expect when running these tests.
This patch (of 5):
In order to integrate debugging tools like KASAN into the KUnit framework, add KUnit struct to the current task to keep track of the current KUnit test.
---
include/linux/sched.h | 4 ++++ 1 file changed, 4 insertions(+)
include/linux/sched.h renamed from include/linux/sched.h~add-kunit-struct-to-current-task +4 -0
@@ -1208,6 +1208,10 @@ struct task_struct {#endif #endif +#if IS_ENABLED (CONFIG_KUNIT ) +struct kunit *kunit_test; +#endif +#ifdef CONFIG_FUNCTION_GRAPH_TRACER /* Index of current stored address in ret_stack: */ int curr_ret_stack;
_
Andrew Morton
maintainerPATCH 121/181 KUnit: KASAN Integration
In reply to Andrew MortonFrom: Patricia Alfonso <[email protected]>
Subject: KUnit: KASAN Integration
Integrate KASAN into KUnit testing framework.
- Fail tests when KASAN reports an error that is not expected
- Use KUNIT_EXPECT_KASAN_FAIL to expect a KASAN error in KASAN tests
- Expected KASAN reports pass tests and are still printed when run without kunit_tool (kunit_tool still bypasses the report due to the test passing)
- KUnit struct in current task used to keep track of the current test from KASAN code
Make use of "[PATCH v3 kunit-next 1/2] kunit: generalize kunit_resource API beyond allocated resources" and "[PATCH v3 kunit-next 2/2] kunit: add support for named resources" from Alan Maguire [1]
- A named resource is added to a test when a KASAN report is expected
- This resource contains a struct for kasan_data containing booleans representing if a KASAN report is expected and if a KASAN report is found
[1] (https://lore.kernel.org/linux-kselftest/[email protected]/T/#t)
---
include/kunit/test.h | 5 ++++ include/linux/kasan.h | 6 +++++ lib/kunit/test.c | 13 ++++++----- lib/test_kasan.c | 47 ++++++++++++++++++++++++++++++++++++++-- mm/kasan/report.c | 32 +++++++++++++++++++++++++++ 5 files changed, 96 insertions(+), 7 deletions(-)
include/kunit/test.h renamed from include/kunit/test.h~kunit-kasan-integration +5 -0
@@ -224,6 +224,11 @@ struct kunit {struct list_head resources;/* Protected by lock. */ }; +static inline void kunit_set_failure (struct kunit *test) +{ +WRITE_ONCE (test->success, false); +} +void kunit_init_test (struct kunit *test,const char *name,char *log);int kunit_run_tests (struct kunit_suite *suite);
include/linux/kasan.h renamed from include/linux/kasan.h~kunit-kasan-integration +6 -0
@@ -14,6 +14,12 @@ struct task_struct;#include <linux/pgtable.h> #include <asm/kasan.h> +/* kasan_data struct is used in KUnit tests for KASAN expected failures */ +struct kunit_kasan_expectation { +bool report_expected; +bool report_found; +}; +extern unsigned char kasan_early_shadow_page[PAGE_SIZE ];extern pte_t kasan_early_shadow_pte[PTRS_PER_PTE ];extern pmd_t kasan_early_shadow_pmd[PTRS_PER_PMD ];
lib/kunit/test.c renamed from lib/kunit/test.c~kunit-kasan-integration +8 -5
@@ -10,16 +10,12 @@#include <linux/kernel.h> #include <linux/kref.h> #include <linux/sched/debug.h> +#include <linux/sched.h> #include "debugfs.h" #include "string-stream.h" #include "try-catch-impl.h" -static void kunit_set_failure (struct kunit *test) -{ -WRITE_ONCE (test->success, false); -} -static void kunit_print_tap_version (void ) {static bool kunit_has_printed_tap_version;
@@ -288,6 +284,10 @@ static void kunit_try_run_case(void *datstruct kunit_suite *suite = ctx->suite;struct kunit_case *test_case = ctx->test_case; +#if (IS_ENABLED (CONFIG_KASAN ) &&IS_ENABLED (CONFIG_KUNIT )) + current->kunit_test = test; +#endif /* IS_ENABLED(CONFIG_KASAN) && IS_ENABLED(CONFIG_KUNIT) */ +/* * kunit_run_case_internal may encounter a fatal error; if it does, * abort will be called, this thread will exit, and finally the parent
@@ -602,6 +602,9 @@ void kunit_cleanup(struct kunit *test)spin_unlock (&test->lock);kunit_remove_resource (test, res); } +#if (IS_ENABLED (CONFIG_KASAN ) &&IS_ENABLED (CONFIG_KUNIT )) + current->kunit_test =NULL ; +#endif /* IS_ENABLED(CONFIG_KASAN) && IS_ENABLED(CONFIG_KUNIT)*/ }EXPORT_SYMBOL_GPL (kunit_cleanup);
lib/test_kasan.c renamed from lib/test_kasan.c~kunit-kasan-integration +45 -2
65 lines not on this page · raw message
mm/kasan/report.c renamed from mm/kasan/report.c~kunit-kasan-integration +32 -0
@@ -33,6 +33,8 @@#include <asm/sections.h> +#include <kunit/test.h> +#include "kasan.h" #include "../slab.h"
@@ -464,12 +466,37 @@ static bool report_enabled(void)return !test_and_set_bit (KASAN_BIT_REPORTED , &kasan_flags); } +#if IS_ENABLED (CONFIG_KUNIT ) +static void kasan_update_kunit_status (struct kunit *cur_test) +{ +struct kunit_resource *resource; +struct kunit_kasan_expectation *kasan_data; + + resource =kunit_find_named_resource (cur_test,"kasan_data" ); + +if (!resource) { +kunit_set_failure (cur_test); +return ; + } + + kasan_data = (struct kunit_kasan_expectation *)resource->data; + kasan_data->report_found = true; +kunit_put_resource (resource); +} +#endif /* IS_ENABLED(CONFIG_KUNIT) */ +void kasan_report_invalid_free (void *object,unsigned long ip) {unsigned long flags;u8 tag =get_tag (object); object =reset_tag (object); + +#if IS_ENABLED (CONFIG_KUNIT ) +if (current->kunit_test) +kasan_update_kunit_status (current->kunit_test); +#endif /* IS_ENABLED(CONFIG_KUNIT) */ +start_report (&flags);pr_err ("BUG: KASAN: double-free or invalid-free in %pS\n" , (void *)ip);print_tags (tag, object);
@@ -488,6 +515,11 @@ static void __kasan_report(unsigned longvoid *untagged_addr;unsigned long flags; +#if IS_ENABLED (CONFIG_KUNIT ) +if (current->kunit_test) +kasan_update_kunit_status (current->kunit_test); +#endif /* IS_ENABLED(CONFIG_KUNIT) */ +disable_trace_on_warning (); tagged_addr = (void *)addr;
_
Andrew Morton
maintainerPATCH 122/181 KASAN: port KASAN Tests to KUnit
In reply to Andrew MortonFrom: Patricia Alfonso <[email protected]>
Subject: KASAN: port KASAN Tests to KUnit
Transfer all previous tests for KASAN to KUnit so they can be run more easily. Using kunit_tool, developers can run these tests with their other KUnit tests and see "pass" or "fail" with the appropriate KASAN report instead of needing to parse each KASAN report to test KASAN functionalities. All KASAN reports are still printed to dmesg.
Stack tests do not work properly when KASAN_STACK is enabled so those tests use a check for "if IS_ENABLED(CONFIG_KASAN_STACK)" so they only run if stack instrumentation is enabled. If KASAN_STACK is not enabled, KUnit will print a statement to let the user know this test was not run with KASAN_STACK enabled.
copy_user_test and kasan_rcu_uaf cannot be run in KUnit so there is a separate test file for those tests, which can be run as before as a module.
---
lib/Kconfig.kasan | 22 - lib/Makefile | 4 lib/test_kasan.c | 685 ++++++++++++++------------------------ lib/test_kasan_module.c | 111 ++++++ 4 files changed, 385 insertions(+), 437 deletions(-)
lib/Kconfig.kasan renamed from lib/Kconfig.kasan~kasan-port-kasan-tests-to-kunit +17 -5
@@ -166,12 +166,24 @@ config KASAN_VMALLOC for KASAN to detect more sorts of errors (and to support vmapped stacks), but at the cost of higher memory usage. -config TEST_KASAN - tristate "Module for testing KASAN for bug detection" - depends on m +config KASAN_KUNIT_TEST + tristate "KUnit-compatible tests of KASAN bug detection capabilities" if !KUNIT_ALL_TESTS + depends on KASAN && KUNIT + default KUNIT_ALL_TESTS help - This is a test module doing various nasty things like - out of bounds accesses, use after free. It is useful for testing + This is a KUnit test suite doing various nasty things like + out of bounds and use after free accesses. It is useful for testing kernel debugging features like KASAN. + For more information on KUnit and unit tests in general, please refer + to the KUnit documentation in Documentation/dev-tools/kunit + +config TEST_KASAN_MODULE + tristate "KUnit-incompatible tests of KASAN bug detection capabilities" + depends on m && KASAN + help + This is a part of the KASAN test suite that is incompatible with + KUnit. Currently includes tests that do bad copy_from/to_user + accesses. + endif # KASAN
lib/Makefile renamed from lib/Makefile~kasan-port-kasan-tests-to-kunit +3 -1
@@ -65,9 +65,11 @@ CFLAGS_test_bitops.o += -Werror obj-$(CONFIG_TEST_SYSCTL) += test_sysctl.o obj-$(CONFIG_TEST_HASH) += test_hash.o test_siphash.o obj-$(CONFIG_TEST_IDA) += test_ida.o -obj-$(CONFIG_TEST_KASAN) += test_kasan.o +obj-$(CONFIG_KASAN_KUNIT_TEST) += test_kasan.o CFLAGS_test_kasan.o += -fno-builtin CFLAGS_test_kasan.o += $(call cc-disable-warning, vla) +obj-$(CONFIG_TEST_KASAN_MODULE) += test_kasan_module.o +CFLAGS_test_kasan_module.o += -fno-builtin obj-$(CONFIG_TEST_UBSAN) += test_ubsan.o CFLAGS_test_ubsan.o += $(call cc-disable-warning, vla) UBSAN_SANITIZE_test_ubsan.o := y
lib/test_kasan.c renamed from lib/test_kasan.c~kasan-port-kasan-tests-to-kunit +254 -431
1,109 lines not on this page · raw message
lib/test_kasan_module.c new file +111 -0
111 lines not on this page · raw message
_
Andrew Morton
maintainerPATCH 123/181 KASAN: Testing Documentation
In reply to Andrew MortonFrom: Patricia Alfonso <[email protected]>
Subject: KASAN: Testing Documentation
Include documentation on how to test KASAN using CONFIG_TEST_KASAN_KUNIT and CONFIG_TEST_KASAN_MODULE.
---
Documentation/dev-tools/kasan.rst | 70 ++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+)
Documentation/dev-tools/kasan.rst renamed from Documentation/dev-tools/kasan.rst~kasan-testing-documentation +70 -0
@@ -281,3 +281,73 @@ unmapped. This will require changes in a This allows ``VMAP_STACK`` support on x86, and can simplify support of architectures that do not have a fixed module region. + +CONFIG_KASAN_KUNIT_TEST & CONFIG_TEST_KASAN_MODULE +-------------------------------------------------- + +``CONFIG_KASAN_KUNIT_TEST`` utilizes the KUnit Test Framework for testing. +This means each test focuses on a small unit of functionality and +there are a few ways these tests can be run. + +Each test will print the KASAN report if an error is detected and then +print the number of the test and the status of the test: + +pass:: + + ok 28 - kmalloc_double_kzfree +or, if kmalloc failed:: + + # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163 + Expected ptr is not null, but is + not ok 4 - kmalloc_large_oob_right +or, if a KASAN report was expected, but not found:: + + # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 + Expected kasan_data->report_expected == kasan_data->report_found, but + kasan_data->report_expected == 1 + kasan_data->report_found == 0 + not ok 28 - kmalloc_double_kzfree + +All test statuses are tracked as they run and an overall status will +be printed at the end:: + + ok 1 - kasan + +or:: + + not ok 1 - kasan + +(1) Loadable Module +~~~~~~~~~~~~~~~~~~~~ + +With ``CONFIG_KUNIT`` enabled, ``CONFIG_KASAN_KUNIT_TEST`` can be built as +a loadable module and run on any architecture that supports KASAN +using something like insmod or modprobe. The module is called ``test_kasan``. + +(2) Built-In +~~~~~~~~~~~~~ + +With ``CONFIG_KUNIT`` built-in, ``CONFIG_KASAN_KUNIT_TEST`` can be built-in +on any architecure that supports KASAN. These and any other KUnit +tests enabled will run and print the results at boot as a late-init +call. + +(3) Using kunit_tool +~~~~~~~~~~~~~~~~~~~~~ + +With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, we can also +use kunit_tool to see the results of these along with other KUnit +tests in a more readable way. This will not print the KASAN reports +of tests that passed. Use `KUnit documentation <https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html>`_ for more up-to-date +information on kunit_tool. + +.. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html + +``CONFIG_TEST_KASAN_MODULE`` is a set of KASAN tests that could not be +converted to KUnit. These tests can be run only as a module with +``CONFIG_TEST_KASAN_MODULE`` built as a loadable module and +``CONFIG_KASAN`` built-in. The type of error expected and the +function being run is printed before the expression expected to give +an error. Then the error is printed, if found, and that test +should be interpretted to pass only if the error was the one expected +by the test.
_
Andrew Morton
maintainerPATCH 124/181 mm: kasan: do not panic if both panic_on_warn and kasan_multishot set
In reply to Andrew MortonFrom: David Gow <[email protected]>
Subject: mm: kasan: do not panic if both panic_on_warn and kasan_multishot set
KASAN errors will currently trigger a panic when panic_on_warn is set. This renders kasan_multishot useless, as further KASAN errors won't be reported if the kernel has already paniced. By making kasan_multishot disable this behaviour for KASAN errors, we can still have the benefits of panic_on_warn for non-KASAN warnings, yet be able to use kasan_multishot.
This is particularly important when running KASAN tests, which need to trigger multiple KASAN errors: previously these would panic the system if panic_on_warn was set, now they can run (and will panic the system should non-KASAN warnings show up).
---
mm/kasan/report.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/kasan/report.c renamed from mm/kasan/report.c~mm-kasan-do-not-panic-if-both-panic_on_warn-and-kasan_multishot-set +1 -1
@@ -95,7 +95,7 @@ static void end_report(unsigned long *flpr_err ("==================================================================\n" );add_taint (TAINT_BAD_PAGE ,LOCKDEP_NOW_UNRELIABLE );spin_unlock_irqrestore (&report_lock, *flags); -if (panic_on_warn) { +if (panic_on_warn && !test_bit (KASAN_BIT_MULTI_SHOT , &kasan_flags)) {/* * This thread may hit another WARN() in the panic path. * Resetting this prevents additional WARN() from panicking the
_
Andrew Morton
maintainerPATCH 125/181 mm/page_alloc: tweak comments in has_unmovable_pages()
In reply to Andrew MortonFrom: David Hildenbrand <[email protected]>
Subject: mm/page_alloc: tweak comments in has_unmovable_pages()
Patch series "mm / virtio-mem: support ZONE_MOVABLE", v5.
When introducing virtio-mem, the semantics of ZONE_MOVABLE were rather unclear, which is why we special-cased ZONE_MOVABLE such that partially plugged blocks would never end up in ZONE_MOVABLE.
Now that the semantics are much clearer (and are documented in patch #6), let's support partially plugged memory blocks in ZONE_MOVABLE, allowing partially plugged memory blocks to be online to ZONE_MOVABLE and also unplugging from such memory blocks. This avoids surprises when onlining of memory blocks suddenly fails, just because they are not completely populated by virtio-mem (yet).
This is especially helpful for testing, but also paves the way for virtio-mem optimizations, allowing more memory to get reliably unplugged.
Cleanup has_unmovable_pages() and set_migratetype_isolate(), providing better documentation of how ZONE_MOVABLE interacts with different kind of unmovable pages (memory offlining vs. alloc_contig_range()).
This patch (of 6):
Let's move the split comment regarding bootmem allocations and memory holes, especially in the context of ZONE_MOVABLE, to the PageReserved() check.
---
mm/page_alloc.c | 22 ++++++---------------- 1 file changed, 6 insertions(+), 16 deletions(-)
mm/page_alloc.c renamed from mm/page_alloc.c~mm-page_alloc-tweak-comments-in-has_unmovable_pages +6 -16
@@ -8235,14 +8235,6 @@ struct page *has_unmovable_pages(structunsigned long iter =0 ;unsigned long pfn =page_to_pfn (page); -/* -* TODO we could make this much more efficient by not checking every -* page in the range if we know all of them are in MOVABLE_ZONE and -* that the movable zone guarantees that pages are migratable but -* the later is not the case right now unfortunatelly. E.g. movablecore -* can still lead to having bootmem allocations in zone_movable. -*/ -if (is_migrate_cma_page (page)) {/* * CMA allocations (alloc_contig_range) really need to mark
@@ -8261,6 +8253,12 @@ struct page *has_unmovable_pages(struct page =pfn_to_page (pfn + iter); +/* +* Both, bootmem allocations and memory holes are marked +* PG_reserved and are unmovable. We can even have unmovable +* allocations inside ZONE_MOVABLE, for example when +* specifying "movablecore". +*/ if (PageReserved (page))return page;
@@ -8334,14 +8332,6 @@ struct page *has_unmovable_pages(struct* it. But now, memory offline itself doesn't call * shrink_node_slabs() and it still to be fixed. */ -/* -* If the page is not RAM, page_count()should be 0. -* we don't need more check. This is an _used_ not-movable page. -* -* The problematic thing here is PG_reserved pages. PG_reserved -* is set to both of a memory hole page and a _used_ kernel -* page at boot. -*/ return page; }return NULL ;
_
Andrew Morton
maintainerPATCH 126/181 mm/page_isolation: exit early when pageblock is isolated in set_migratetype_isolate()
In reply to Andrew MortonFrom: David Hildenbrand <[email protected]>
Subject: mm/page_isolation: exit early when pageblock is isolated in set_migratetype_isolate()
Right now, if we have two isolations racing on a pageblock that's in the MOVABLE zone, we would trigger the WARN_ON_ONCE(). Let's just return directly, simplifying error handling.
The change was introduced in commit 3d680bdf60a5 ("mm/page_isolation: fix potential warning from user"). As far as I can see, we currently don't have alloc_contig_range() users that use the ZONE_MOVABLE (anymore), so it's currently more a cleanup and a preparation for the future than a fix.
---
mm/page_isolation.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
mm/page_isolation.c renamed from mm/page_isolation.c~mm-page_isolation-exit-early-when-pageblock-is-isolated-in-set_migratetype_isolate +5 -4
@@ -29,10 +29,12 @@ static int set_migratetype_isolate(struc/* * We assume the caller intended to SET migrate type to isolate. * If it is already set, then someone else must have raced and -* set it before us. Return -EBUSY +* set it before us. */ -if (is_migrate_isolate_page (page)) - goto out; +if (is_migrate_isolate_page (page)) { +spin_unlock_irqrestore (&zone->lock, flags); +return -EBUSY ; + }/* * FIXME: Now, memory hotplug doesn't call shrink_slab() by itself.
@@ -52,7 +54,6 @@ static int set_migratetype_isolate(struc ret =0 ; } -out:spin_unlock_irqrestore (&zone->lock, flags);if (!ret) {drain_all_pages (zone);
_
Andrew Morton
maintainerPATCH 127/181 mm/page_isolation: drop WARN_ON_ONCE() in set_migratetype_isolate()
In reply to Andrew MortonFrom: David Hildenbrand <[email protected]>
Subject: mm/page_isolation: drop WARN_ON_ONCE() in set_migratetype_isolate()
Inside has_unmovable_pages(), we have a comment describing how unmovable data could end up in ZONE_MOVABLE - via "movablecore". Also, besides checking if the first page in the pageblock is reserved, we don't perform any further checks in case of ZONE_MOVABLE.
In case of memory offlining, we set REPORT_FAILURE, properly dump_page() the page and handle the error gracefully. alloc_contig_pages() users currently never allocate from ZONE_MOVABLE. E.g., hugetlb uses alloc_contig_pages() for the allocation of gigantic pages only, which will never end up on the MOVABLE zone (see htlb_alloc_mask()).
---
mm/page_isolation.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-)
mm/page_isolation.c renamed from mm/page_isolation.c~mm-page_isolation-drop-warn_on_once-in-set_migratetype_isolate +6 -9
@@ -57,15 +57,12 @@ static int set_migratetype_isolate(strucspin_unlock_irqrestore (&zone->lock, flags);if (!ret) {drain_all_pages (zone); - }else { -WARN_ON_ONCE (zone_idx (zone) ==ZONE_MOVABLE ); - -if ((isol_flags &REPORT_FAILURE ) && unmovable) -/* -* printk() with zone->lock held will likely trigger a -* lockdep splat, so defer it here. -*/ -dump_page (unmovable,"unmovable page" ); + }else if ((isol_flags &REPORT_FAILURE ) && unmovable) { +/* +* printk() with zone->lock held will likely trigger a +* lockdep splat, so defer it here. +*/ +dump_page (unmovable,"unmovable page" ); }return ret;
_
Andrew Morton
maintainerPATCH 128/181 mm/page_isolation: cleanup set_migratetype_isolate()
In reply to Andrew MortonFrom: David Hildenbrand <[email protected]>
Subject: mm/page_isolation: cleanup set_migratetype_isolate()
Let's clean it up a bit, simplifying the exit paths.
---
mm/page_isolation.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)
mm/page_isolation.c renamed from mm/page_isolation.c~mm-page_isolation-cleanup-set_migratetype_isolate +7 -10
@@ -17,12 +17,9 @@static int set_migratetype_isolate (struct page *page,int migratetype,int isol_flags) { -struct page *unmovable =NULL ; -struct zone *zone; +struct zone *zone =page_zone (page); +struct page *unmovable;unsigned long flags; -int ret = -EBUSY ; - - zone =page_zone (page);spin_lock_irqsave (&zone->lock, flags);
@@ -51,13 +48,13 @@ static int set_migratetype_isolate(struc NULL);__mod_zone_freepage_state (zone, -nr_pages, mt); - ret =0 ; +spin_unlock_irqrestore (&zone->lock, flags); +drain_all_pages (zone); +return 0 ; }spin_unlock_irqrestore (&zone->lock, flags); -if (!ret) { -drain_all_pages (zone); - }else if ((isol_flags &REPORT_FAILURE ) && unmovable) { +if (isol_flags &REPORT_FAILURE ) {/* * printk() with zone->lock held will likely trigger a * lockdep splat, so defer it here.
@@ -65,7 +62,7 @@ static int set_migratetype_isolate(strucdump_page (unmovable,"unmovable page" ); } -return ret; +return -EBUSY ; }static void unset_migratetype_isolate (struct page *page,unsigned migratetype)
_
Andrew Morton
maintainerPATCH 129/181 virtio-mem: don't special-case ZONE_MOVABLE
In reply to Andrew MortonFrom: David Hildenbrand <[email protected]>
Subject: virtio-mem: don't special-case ZONE_MOVABLE
When introducing virtio-mem, the semantics of ZONE_MOVABLE were rather unclear, which is why we special-cased ZONE_MOVABLE such that partially plugged blocks would never end up in ZONE_MOVABLE.
Now that the semantics are much clearer (and will be documented in a follow-up patch including the new virtio-mem behavior), let's allow to online partially plugged memory blocks to ZONE_MOVABLE and also consider memory blocks that were onlined to ZONE_MOVABLE when unplugging memory. While unplugged memory pages are, in general, unmovable, they can be skipped when offlining memory.
virtio-mem only unplugs fairly big chunks (in the megabyte range) and rather tries to shrink the memory region than randomly choosing memory. In theory, if all other pages in the movable zone would be movable, virtio-mem would only shrink that zone and not create any kind of fragmentation.
In the future, we might want to remember the zone again and use the information when (un)plugging memory. For now, let's keep it simple.
Note: Support for defragmentation is planned, to deal with fragmentation after unplug due to memory chunks within memory blocks that could not get unplugged before (e.g., somebody pinning pages within ZONE_MOVABLE for a longer time).
---
drivers/virtio/virtio_mem.c | 47 +++++----------------------------- 1 file changed, 8 insertions(+), 39 deletions(-)
drivers/virtio/virtio_mem.c renamed from drivers/virtio/virtio_mem.c~virtio-mem-dont-special-case-zone_movable +8 -39
@@ -36,18 +36,10 @@ enum virtio_mem_mb_state {VIRTIO_MEM_MB_STATE_OFFLINE ,/* Partially plugged, fully added to Linux, offline. */ VIRTIO_MEM_MB_STATE_OFFLINE_PARTIAL , -/* Fully plugged, fully added to Linux, online (!ZONE_MOVABLE). */ +/* Fully plugged, fully added to Linux, online. */ VIRTIO_MEM_MB_STATE_ONLINE , -/* Partially plugged, fully added to Linux, online (!ZONE_MOVABLE). */ +/* Partially plugged, fully added to Linux, online. */ VIRTIO_MEM_MB_STATE_ONLINE_PARTIAL , -/* -* Fully plugged, fully added to Linux, online (ZONE_MOVABLE). -* We are not allowed to allocate (unplug) parts of this block that -* are not movable (similar to gigantic pages). We will never allow -* to online OFFLINE_PARTIAL to ZONE_MOVABLE (as they would contain -* unmovable parts). -*/ -VIRTIO_MEM_MB_STATE_ONLINE_MOVABLE , VIRTIO_MEM_MB_STATE_COUNT };
@@ -526,21 +518,10 @@ static bool virtio_mem_owned_mb(struct v }static int virtio_mem_notify_going_online (struct virtio_mem *vm, -unsigned long mb_id, -enum zone_type zone) +unsigned long mb_id) {switch (virtio_mem_mb_get_state (vm, mb_id)) {case VIRTIO_MEM_MB_STATE_OFFLINE_PARTIAL : -/* -* We won't allow to online a partially plugged memory block -* to the MOVABLE zone - it would contain unmovable parts. -*/ -if (zone ==ZONE_MOVABLE ) { -dev_warn_ratelimited (&vm->vdev->dev, -"memory block has holes, MOVABLE not supported\n" ); -return NOTIFY_BAD ; - } -return NOTIFY_OK ;case VIRTIO_MEM_MB_STATE_OFFLINE :return NOTIFY_OK ;default :
@@ -560,7 +541,6 @@ static void virtio_mem_notify_offline(st VIRTIO_MEM_MB_STATE_OFFLINE_PARTIAL);break ;case VIRTIO_MEM_MB_STATE_ONLINE : -case VIRTIO_MEM_MB_STATE_ONLINE_MOVABLE :virtio_mem_mb_set_state (vm, mb_id,VIRTIO_MEM_MB_STATE_OFFLINE );break ;
@@ -579,24 +559,17 @@ static void virtio_mem_notify_offline(stvirtio_mem_retry (vm); } -static void virtio_mem_notify_online (struct virtio_mem *vm,unsigned long mb_id, -enum zone_type zone) +static void virtio_mem_notify_online (struct virtio_mem *vm,unsigned long mb_id) {unsigned long nb_offline;switch (virtio_mem_mb_get_state (vm, mb_id)) {case VIRTIO_MEM_MB_STATE_OFFLINE_PARTIAL : -BUG_ON (zone ==ZONE_MOVABLE );virtio_mem_mb_set_state (vm, mb_id,VIRTIO_MEM_MB_STATE_ONLINE_PARTIAL );break ;case VIRTIO_MEM_MB_STATE_OFFLINE : -if (zone ==ZONE_MOVABLE ) -virtio_mem_mb_set_state (vm, mb_id, -VIRTIO_MEM_MB_STATE_ONLINE_MOVABLE ); -else -virtio_mem_mb_set_state (vm, mb_id, -VIRTIO_MEM_MB_STATE_ONLINE ); +virtio_mem_mb_set_state (vm, mb_id,VIRTIO_MEM_MB_STATE_ONLINE );break ;default :BUG ();
@@ -675,7 +648,6 @@ static int virtio_mem_memory_notifier_cbconst unsigned long start =PFN_PHYS (mhp->start_pfn);const unsigned long size =PFN_PHYS (mhp->nr_pages);const unsigned long mb_id =virtio_mem_phys_to_mb_id (start); -enum zone_type zone;int rc =NOTIFY_OK ;if (!virtio_mem_overlaps_range (vm, start, size))
@@ -717,8 +689,7 @@ static int virtio_mem_memory_notifier_cbbreak ; } vm->hotplug_active = true; - zone =page_zonenum (pfn_to_page (mhp->start_pfn)); - rc =virtio_mem_notify_going_online (vm, mb_id, zone); + rc =virtio_mem_notify_going_online (vm, mb_id);break ;case MEM_OFFLINE :virtio_mem_notify_offline (vm, mb_id);
@@ -726,8 +697,7 @@ static int virtio_mem_memory_notifier_cbmutex_unlock (&vm->hotplug_mutex);break ;case MEM_ONLINE : - zone =page_zonenum (pfn_to_page (mhp->start_pfn)); -virtio_mem_notify_online (vm, mb_id, zone); +virtio_mem_notify_online (vm, mb_id); vm->hotplug_active = false;mutex_unlock (&vm->hotplug_mutex);break ;
@@ -1906,8 +1876,7 @@ static void virtio_mem_remove(struct virif (vm->nb_mb_state[VIRTIO_MEM_MB_STATE_OFFLINE ] || vm->nb_mb_state[VIRTIO_MEM_MB_STATE_OFFLINE_PARTIAL ] || vm->nb_mb_state[VIRTIO_MEM_MB_STATE_ONLINE ] || - vm->nb_mb_state[VIRTIO_MEM_MB_STATE_ONLINE_PARTIAL ] || - vm->nb_mb_state[VIRTIO_MEM_MB_STATE_ONLINE_MOVABLE ]) { + vm->nb_mb_state[VIRTIO_MEM_MB_STATE_ONLINE_PARTIAL ]) {dev_warn (&vdev->dev,"device still has system memory added\n" ); }else {virtio_mem_delete_resource (vm);
_
Andrew Morton
maintainerPATCH 130/181 mm: document semantics of ZONE_MOVABLE
In reply to Andrew MortonFrom: David Hildenbrand <[email protected]>
Subject: mm: document semantics of ZONE_MOVABLE
Let's document what ZONE_MOVABLE means, how it's used, and which special cases we have regarding unmovable pages (memory offlining vs. migration / allocations).
---
include/linux/mmzone.h | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
include/linux/mmzone.h renamed from include/linux/mmzone.h~mm-document-semantics-of-zone_movable +35 -0
@@ -396,6 +396,41 @@ enum zone_type {*/ ZONE_HIGHMEM,#endif +/* +* ZONE_MOVABLE is similar to ZONE_NORMAL, except that it contains +* movable pages with few exceptional cases described below. Main use +* cases for ZONE_MOVABLE are to make memory offlining/unplug more +* likely to succeed, and to locally limit unmovable allocations - e.g., +* to increase the number of THP/huge pages. Notable special cases are: +* +* 1. Pinned pages: (long-term) pinning of movable pages might +* essentially turn such pages unmovable. Memory offlining might +* retry a long time. +* 2. memblock allocations: kernelcore/movablecore setups might create +* situations where ZONE_MOVABLE contains unmovable allocations +* after boot. Memory offlining and allocations fail early. +* 3. Memory holes: kernelcore/movablecore setups might create very rare +* situations where ZONE_MOVABLE contains memory holes after boot, +* for example, if we have sections that are only partially +* populated. Memory offlining and allocations fail early. +* 4. PG_hwpoison pages: while poisoned pages can be skipped during +* memory offlining, such pages cannot be allocated. +* 5. Unmovable PG_offline pages: in paravirtualized environments, +* hotplugged memory blocks might only partially be managed by the +* buddy (e.g., via XEN-balloon, Hyper-V balloon, virtio-mem). The +* parts not manged by the buddy are unmovable PG_offline pages. In +* some cases (virtio-mem), such pages can be skipped during +* memory offlining, however, cannot be moved/allocated. These +* techniques might use alloc_contig_range() to hide previously +* exposed pages from the buddy again (e.g., to implement some sort +* of memory unplug in virtio-mem). +* +* In general, no unmovable allocations that degrade memory offlining +* should end up in ZONE_MOVABLE. Allocators (like alloc_contig_range()) +* have to expect that migrating pages in ZONE_MOVABLE can fail (even +* if has_unmovable_pages() states that there are no unmovable pages, +* there can be false negatives). +*/ ZONE_MOVABLE,#ifdef CONFIG_ZONE_DEVICE ZONE_DEVICE,
_
Andrew Morton
maintainerPATCH 131/181 mm, isolation: avoid checking unmovable pages across pageblock boundary
In reply to Andrew MortonFrom: Li Xinhai <[email protected]>
Subject: mm, isolation: avoid checking unmovable pages across pageblock boundary
In has_unmovable_pages(), the page parameter would not always be the first page within a pageblock (see how the page pointer is passed in from start_isolate_page_range() after call __first_valid_page()), so that would cause checking unmovable pages span two pageblocks.
After this patch, the checking is enforced within one pageblock no matter the page is first one or not, and obey the semantics of this function.
This issue is found by code inspection.
Michal said "this might lead to false negatives when an unrelated block would cause an isolation failure".
---
mm/page_alloc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mm/page_alloc.c renamed from mm/page_alloc.c~mm-isolation-avoid-checking-unmovable-pages-across-pageblock-boundary +2 -1
@@ -8234,6 +8234,7 @@ struct page *has_unmovable_pages(struct {unsigned long iter =0 ;unsigned long pfn =page_to_pfn (page); +unsigned long offset = pfn % pageblock_nr_pages;if (is_migrate_cma_page (page)) {/*
@@ -8247,7 +8248,7 @@ struct page *has_unmovable_pages(structreturn page; } -for (; iter < pageblock_nr_pages; iter++) { +for (; iter < pageblock_nr_pages - offset; iter++) {if (!pfn_valid_within (pfn + iter))continue ;
_
Andrew Morton
maintainerPATCH 132/181 mm/page_alloc.c: clean code by removing unnecessary initialization
In reply to Andrew MortonFrom: Mateusz Nosek <[email protected]>
Subject: mm/page_alloc.c: clean code by removing unnecessary initialization
Previously variable 'tmp' was initialized, but was not read later before reassigning. So the initialization can be removed.
---
mm/page_alloc.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
mm/page_alloc.c renamed from mm/page_alloc.c~mm-page_allocc-clean-code-by-removing-unnecessary-initialization +1 -3
@@ -5651,7 +5651,6 @@ static int find_next_best_node(int node,int n, val;int min_val =INT_MAX ;int best_node =NUMA_NO_NODE ; -const struct cpumask *tmp =cpumask_of_node (0 );/* Use the local node if we haven't already */ if (!node_isset (node, *used_node_mask)) {
@@ -5672,8 +5671,7 @@ static int find_next_best_node(int node, val += (n < node);/* Give preference to headless and unused nodes */ - tmp =cpumask_of_node (n); -if (!cpumask_empty (tmp)) +if (!cpumask_empty (cpumask_of_node (n))) val +=PENALTY_FOR_NODE_WITH_CPUS ;/* Slight preference for less loaded node */
_
Andrew Morton
maintainerPATCH 133/181 mm/page_alloc.c: micro-optimization remove unnecessary branch
In reply to Andrew MortonFrom: Mateusz Nosek <[email protected]>
Subject: mm/page_alloc.c: micro-optimization remove unnecessary branch
Previously flags check was separated into two separated checks with two separated branches. In case of presence of any of two mentioned flags, the same effect on flow occurs. Therefore checks can be merged and one branch can be avoided.
---
mm/page_alloc.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
mm/page_alloc.c renamed from mm/page_alloc.c~mm-page_allocc-micro-optimization-remove-unnecessary-branch +3 -5
@@ -3986,8 +3986,10 @@ __alloc_pages_may_oom(gfp_t gfp_mask, un* success so it is time to admit defeat. We will skip the OOM killer * because it is very likely that the caller has a more reasonable * fallback than shooting a random task. +* +* The OOM killer may not free memory on a specific node. */ -if (gfp_mask & __GFP_RETRY_MAYFAIL) +if (gfp_mask & (__GFP_RETRY_MAYFAIL | __GFP_THISNODE)) goto out;/* The OOM killer does not needlessly kill tasks for lowmem */ if (ac->highest_zoneidx <ZONE_NORMAL )
@@ -4004,10 +4006,6 @@ __alloc_pages_may_oom(gfp_t gfp_mask, un* failures more gracefully we should just bail out here. */ -/* The OOM killer may not free memory on a specific node */ -if (gfp_mask & __GFP_THISNODE) - goto out; -/* Exhausted what can be done so it's blame time */ if (out_of_memory (&oc) ||WARN_ON_ONCE (gfp_mask & __GFP_NOFAIL)) { *did_some_progress =1 ;
_
Andrew Morton
maintainerPATCH 134/181 mm/page_alloc.c: fix early params garbage value accesses
In reply to Andrew MortonFrom: Mateusz Nosek <[email protected]>
Subject: mm/page_alloc.c: fix early params garbage value accesses
Previously in '__init early_init_on_alloc' and '__init early_init_on_free' the return values from 'kstrtobool' were not handled properly. That caused potential garbage value read from variable 'bool_result'. Introduced patch fixes error handling.
---
mm/page_alloc.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
mm/page_alloc.c renamed from mm/page_alloc.c~mm-page_allocc-fix-early-params-garbage-value-accesses +6 -6
@@ -156,16 +156,16 @@ static int __init early_init_on_alloc(chint ret;bool bool_result; -if (!buf) -return -EINVAL ; ret =kstrtobool (buf, &bool_result); +if (ret) +return ret;if (bool_result &&page_poisoning_enabled ())pr_info ("mem auto-init: CONFIG_PAGE_POISONING is on, will take precedence over init_on_alloc\n" );if (bool_result)static_branch_enable (&init_on_alloc);else static_branch_disable (&init_on_alloc); -return ret; +return 0 ; }early_param ("init_on_alloc" , early_init_on_alloc);
@@ -174,16 +174,16 @@ static int __init early_init_on_free(chaint ret;bool bool_result; -if (!buf) -return -EINVAL ; ret =kstrtobool (buf, &bool_result); +if (ret) +return ret;if (bool_result &&page_poisoning_enabled ())pr_info ("mem auto-init: CONFIG_PAGE_POISONING is on, will take precedence over init_on_free\n" );if (bool_result)static_branch_enable (&init_on_free);else static_branch_disable (&init_on_free); -return ret; +return 0 ; }early_param ("init_on_free" , early_init_on_free);
_
Andrew Morton
maintainerPATCH 135/181 mm/page_alloc.c: clean code by merging two functions
In reply to Andrew MortonFrom: Mateusz Nosek <[email protected]>
Subject: mm/page_alloc.c: clean code by merging two functions
finalise_ac() is just 'epilogue' for 'prepare_alloc_pages'. Therefore there is no need to keep them both so 'finalise_ac' content can be merged into prepare_alloc_pages() code. It would make __alloc_pages_nodemask() cleaner when it comes to readability.
---
mm/page_alloc.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-)
mm/page_alloc.c renamed from mm/page_alloc.c~mm-page_allocc-clean-code-by-merging-two-functions +2 -8
@@ -4838,12 +4838,6 @@ static inline bool prepare_alloc_pages(g *alloc_flags =current_alloc_flags (gfp_mask, *alloc_flags); -return true; -} - -/* Determine whether to spread dirty pages and what the first usable zone */ -static inline void finalise_ac (gfp_t gfp_mask,struct alloc_context *ac) -{/* Dirty zone balancing only done in the fast path */ ac->spread_dirty_pages = (gfp_mask & __GFP_WRITE);
@@ -4854,6 +4848,8 @@ static inline void finalise_ac(gfp_t gfp*/ ac->preferred_zoneref =first_zones_zonelist (ac->zonelist, ac->highest_zoneidx, ac->nodemask); + +return true; }/*
@@ -4882,8 +4878,6 @@ __alloc_pages_nodemask(gfp_t gfp_mask, uif (!prepare_alloc_pages (gfp_mask, order, preferred_nid, nodemask, &ac, &alloc_mask, &alloc_flags))return NULL ; -finalise_ac (gfp_mask, &ac); -/* * Forbid the first pass from falling back to types that fragment * memory until all local zones are considered.
_
Andrew Morton
maintainerPATCH 136/181 mm/page_alloc.c: __perform_reclaim should return 'unsigned long'
In reply to Andrew MortonFrom: Yanfei Xu <[email protected]>
Subject: mm/page_alloc.c: __perform_reclaim should return 'unsigned long'
__perform_reclaim()'s single caller expects it to return 'unsigned long', hence change its return value and a local variable to 'unsigned long'.
---
mm/page_alloc.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
mm/page_alloc.c renamed from mm/page_alloc.c~mm-page_allocc-__perform_reclaim-should-return-unsigned-long +2 -3
@@ -4253,13 +4253,12 @@ EXPORT_SYMBOL_GPL(fs_reclaim_release);#endif /* Perform direct synchronous page reclaim */ -static int +static unsigned long __perform_reclaim (gfp_t gfp_mask,unsigned int order,const struct alloc_context *ac) { -int progress;unsigned int noreclaim_flag; -unsigned long pflags; +unsigned long pflags, progress;cond_resched ();
_
Andrew Morton
maintainerPATCH 137/181 mmzone: clean code by removing unused macro parameter
In reply to Andrew MortonFrom: Mateusz Nosek <[email protected]>
Subject: mmzone: clean code by removing unused macro parameter
Previously 'for_next_zone_zonelist_nodemask' macro parameter 'zlist' was unused so this patch removes it.
---
include/linux/mmzone.h | 2 +- mm/page_alloc.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
include/linux/mmzone.h renamed from include/linux/mmzone.h~mmzone-clean-code-by-removing-unused-macro-parameter +1 -1
@@ -1116,7 +1116,7 @@ static inline struct zoneref *first_zone z = next_zones_zonelist(++z, highidx, nodemask), \ zone = zonelist_zone(z)) -#define for_next_zone_zonelist_nodemask (zone, z, zlist, highidx, nodemask) \ +#define for_next_zone_zonelist_nodemask (zone, z, highidx, nodemask) \ for (zone = z->zone; \ zone; \ z = next_zones_zonelist(++z, highidx, nodemask), \
mm/page_alloc.c renamed from mm/page_alloc.c~mmzone-clean-code-by-removing-unused-macro-parameter +2 -2
@@ -3741,8 +3741,8 @@ retry:*/ no_fallback = alloc_flags &ALLOC_NOFRAGMENT ; z = ac->preferred_zoneref; -for_next_zone_zonelist_nodemask (zone, z, ac->zonelist, - ac->highest_zoneidx, ac->nodemask) { +for_next_zone_zonelist_nodemask (zone, z, ac->highest_zoneidx, + ac->nodemask) {struct page *page;unsigned long mark;
_
Andrew Morton
maintainerPATCH 138/181 mm: move call to compound_head() in release_pages()
In reply to Andrew MortonFrom: Ralph Campbell <[email protected]>
Subject: mm: move call to compound_head() in release_pages()
The function is_huge_zero_page() doesn't call compound_head() to make sure the page pointer is a head page. The call to is_huge_zero_page() in release_pages() is made before compound_head() is called so the test would fail if release_pages() was called with a tail page of the huge_zero_page and put_page_testzero() would be called releasing the page. This is unlikely to be happening in normal use or we would be seeing all sorts of process data corruption when accessing a THP zero page.
Looking at other places where is_huge_zero_page() is called, all seem to only pass a head page so I think the right solution is to move the call to compound_head() in release_pages() to a point before calling is_huge_zero_page().
---
mm/swap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/swap.c renamed from mm/swap.c~mm-move-call-to-compound_head-in-release_pages +1 -1
@@ -889,6 +889,7 @@ void release_pages(struct page **pages, locked_pgdat =NULL ; } + page =compound_head (page);if (is_huge_zero_page (page))continue ;
@@ -910,7 +911,6 @@ void release_pages(struct page **pages, } } - page =compound_head (page);if (!put_page_testzero (page))continue ;
_
Andrew Morton
maintainerPATCH 139/181 mm/page_alloc.c: fix freeing non-compound pages
In reply to Andrew MortonFrom: "Matthew Wilcox (Oracle)" <[email protected]>
Subject: mm/page_alloc.c: fix freeing non-compound pages
Here is a very rare race which leaks memory:
Page P0 is allocated to the page cache. Page P1 is free.
Thread A Thread B Thread Cfind_get_entry():xas_load() returns P0
Removes P0 from page cache P0 finds its buddy P1
alloc_pages(GFP_KERNEL, 1) returns P0 P0 has refcount 1page_cache_get_speculative(P0)
P0 has refcount 2
__free_pages(P0) P0 has refcount 1
put_page(P0)
P1 is not freed
Fix this by freeing all the pages in __free_pages() that won't be freed by the call to put_page(). It's usually not a good idea to split a page, but this is a very unlikely scenario.
---
lib/Kconfig.debug | 9 ++++++++ lib/Makefile | 1 lib/test_free_pages.c | 42 ++++++++++++++++++++++++++++++++++++++++ mm/page_alloc.c | 3 ++ 4 files changed, 55 insertions(+)
lib/Kconfig.debug renamed from lib/Kconfig.debug~page_alloc-fix-freeing-non-compound-pages +9 -0
@@ -2367,6 +2367,15 @@ config TEST_HMM If unsure, say N. +config TEST_FREE_PAGES + tristate "Test freeing pages" + help + Test that a memory leak does not occur due to a race between + freeing a block of pages and a speculative page reference. + Loading this module is safe if your kernel has the bug fixed. + If the bug is not fixed, it will leak gigabytes of memory and + probably OOM your system. + config TEST_FPU tristate "Test floating point operations in kernel space" depends on X86 && !KCOV_INSTRUMENT_ALL
lib/Makefile renamed from lib/Makefile~page_alloc-fix-freeing-non-compound-pages +1 -0
@@ -101,6 +101,7 @@ obj-$(CONFIG_TEST_BLACKHOLE_DEV) += test obj-$(CONFIG_TEST_MEMINIT) += test_meminit.o obj-$(CONFIG_TEST_LOCKUP) += test_lockup.o obj-$(CONFIG_TEST_HMM) += test_hmm.o +obj-$(CONFIG_TEST_FREE_PAGES) += test_free_pages.o # # CFLAGS for compiling floating point code inside the kernel. x86/Makefile turns
lib/test_free_pages.c new file +42 -0
42 lines not on this page · raw message
mm/page_alloc.c renamed from mm/page_alloc.c~page_alloc-fix-freeing-non-compound-pages +3 -0
@@ -4952,6 +4952,9 @@ void __free_pages(struct page *page, uns {if (put_page_testzero (page))free_the_page (page, order); +else if (!PageHead (page)) +while (order-- >0 ) +free_the_page (page + (1 << order), order); }EXPORT_SYMBOL (__free_pages);
_
Andrew Morton
maintainerPATCH 140/181 include/linux/gfp.h: clarify usage of GFP_ATOMIC in !preemptible contexts
In reply to Andrew MortonFrom: Michal Hocko <[email protected]>
Subject: include/linux/gfp.h: clarify usage of GFP_ATOMIC in !preemptible contexts
There is a general understanding that GFP_ATOMIC/GFP_NOWAIT are to be used from atomic contexts. E.g. from within a spin lock or from the IRQ context. This is correct but there are some atomic contexts where the above doesn't hold. One of them would be an NMI context. Page allocator has never supported that and the general fear of this context didn't let anybody to actually even try to use the allocator there. Good, but let's be more specific about that.
Another such a context, and that is where people seem to be more daring, is raw_spin_lock. Mostly because it simply resembles regular spin lock which is supported by the allocator and there is not any implementation difference with !RT kernels in the first place. Be explicit that such a context is not supported by the allocator. The underlying reason is that zone->lock would have to become raw_spin_lock as well and that has turned out to be a problem for RT
(http://lkml.kernel.org/r/[email protected]).
---
include/linux/gfp.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
include/linux/gfp.h renamed from include/linux/gfp.h~mm-clarify-usage-of-gfp_atomic-in-preemptible-contexts +3 -1
@@ -238,7 +238,9 @@ struct vm_area_struct;* %__GFP_FOO flags as necessary. * * %GFP_ATOMIC users can not sleep and need the allocation to succeed. A lower -* watermark is applied to allow access to "atomic reserves" +* watermark is applied to allow access to "atomic reserves". +* The current implementation doesn't support NMI and few other strict +* non-preemptive contexts (e.g. raw_spin_lock). The same applies to %GFP_NOWAIT. * * %GFP_KERNEL is typical for kernel-internal allocations. The caller requires * %ZONE_NORMAL or a lower zone for direct access but can direct reclaim.
_
Andrew Morton
maintainerPATCH 141/181 mm/hugetlb.c: make is_hugetlb_entry_hwpoisoned return bool
In reply to Andrew MortonFrom: Baoquan He <[email protected]>
Subject: mm/hugetlb.c: make is_hugetlb_entry_hwpoisoned return bool
Patch series "mm/hugetlb: Small cleanup and improvement", v2.
This patch (of 3):
Just like its neighbour is_hugetlb_entry_migration() has done.
---
mm/hugetlb.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
mm/hugetlb.c renamed from mm/hugetlb.c~mm-hugetlbc-make-is_hugetlb_entry_hwpoisoned-return-bool +4 -4
@@ -3805,17 +3805,17 @@ bool is_hugetlb_entry_migration(pte_t ptreturn false; } -static int is_hugetlb_entry_hwpoisoned (pte_t pte) +static bool is_hugetlb_entry_hwpoisoned (pte_t pte) {swp_entry_t swp;if (huge_pte_none (pte) ||pte_present (pte)) -return 0 ; +return false; swp =pte_to_swp_entry (pte);if (non_swap_entry (swp) &&is_hwpoison_entry (swp)) -return 1 ; +return true;else -return 0 ; +return false; } int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src,
_
Andrew Morton
maintainerPATCH 142/181 mm/hugetlb.c: remove the unnecessary non_swap_entry()
In reply to Andrew MortonFrom: Baoquan He <[email protected]>
Subject: mm/hugetlb.c: remove the unnecessary non_swap_entry()
If a swap entry tests positive for either is_[migration|hwpoison]_entry(), then its swap_type() is among SWP_MIGRATION_READ, SWP_MIGRATION_WRITE and SWP_HWPOISON. All these types >= MAX_SWAPFILES, exactly what is asserted with non_swap_entry().
So the checking non_swap_entry() in is_hugetlb_entry_migration() and is_hugetlb_entry_hwpoisoned() is redundant.
Let's remove it to optimize code.
---
mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/hugetlb.c renamed from mm/hugetlb.c~mm-hugetlbc-remove-the-unnecessary-non_swap_entry +2 -2
@@ -3799,7 +3799,7 @@ bool is_hugetlb_entry_migration(pte_t ptif (huge_pte_none (pte) ||pte_present (pte))return false; swp =pte_to_swp_entry (pte); -if (non_swap_entry (swp) &&is_migration_entry (swp)) +if (is_migration_entry (swp))return true;else return false;
@@ -3812,7 +3812,7 @@ static bool is_hugetlb_entry_hwpoisoned(if (huge_pte_none (pte) ||pte_present (pte))return false; swp =pte_to_swp_entry (pte); -if (non_swap_entry (swp) &&is_hwpoison_entry (swp)) +if (is_hwpoison_entry (swp))return true;else return false;
_
Andrew Morton
maintainerPATCH 143/181 doc/vm: fix typo in the hugetlb admin documentation
In reply to Andrew MortonFrom: Baoquan He <[email protected]>
Subject: doc/vm: fix typo in the hugetlb admin documentation
Change 'pecify' to 'Specify'.
---
Documentation/admin-guide/mm/hugetlbpage.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Documentation/admin-guide/mm/hugetlbpage.rst renamed from Documentation/admin-guide/mm/hugetlbpage.rst~doc-vm-fix-typo-in-the-hugetlb-admin-documentation +1 -1
@@ -131,7 +131,7 @@ hugepages parameter is preceded by an invalid hugepagesz parameter, it will be ignored. default_hugepagesz - pecify the default huge page size. This parameter can + Specify the default huge page size. This parameter can only be specified once on the command line. default_hugepagesz can optionally be followed by the hugepages parameter to preallocate a specific number of huge pages of default size. The number of default
_
Andrew Morton
maintainerPATCH 144/181 mm/hugetlb: not necessary to coalesce regions recursively
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/hugetlb: not necessary to coalesce regions recursively
Patch series "mm/hugetlb: code refine and simplification", v4.
Following are some cleanups for hugetlb. Simple testing with tools/testing/selftests/vm/map_hugetlb passes.
This patch (of 7):
Per my understanding, we keep the regions ordered and would always coalesce regions properly. So the task to keep this property is just to coalesce its neighbour.
Let's simplify this.
---
mm/hugetlb.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
mm/hugetlb.c renamed from mm/hugetlb.c~mm-hugetlb-not-necessary-to-coalesce-regions-recursively +1 -5
@@ -309,8 +309,7 @@ static void coalesce_file_region(structlist_del (&rg->link);kfree (rg); -coalesce_file_region (resv, prg); -return ; + rg = prg; } nrg =list_next_entry (rg, link);
@@ -320,9 +319,6 @@ static void coalesce_file_region(structlist_del (&rg->link);kfree (rg); - -coalesce_file_region (resv, nrg); -return ; } }
_
Andrew Morton
maintainerPATCH 145/181 mm/hugetlb: remove VM_BUG_ON(!nrg) in get_file_region_entry_from_cache()
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/hugetlb: remove VM_BUG_ON(!nrg) in get_file_region_entry_from_cache()
We are sure to get a valid file_region, otherwise the VM_BUG_ON(resv->region_cache_count <= 0) at the very beginning would be triggered.
Let's remove the redundant one.
---
mm/hugetlb.c | 1 - 1 file changed, 1 deletion(-)
mm/hugetlb.c renamed from mm/hugetlb.c~mm-hugetlb-remove-vm_bug_onnrg-in-get_file_region_entry_from_cache +0 -1
@@ -240,7 +240,6 @@ get_file_region_entry_from_cache(struct resv->region_cache_count--; nrg = list_first_entry(&resv->region_cache, struct file_region, link); -VM_BUG_ON (!nrg);list_del (&nrg->link); nrg->from = from;
_
Andrew Morton
maintainerPATCH 146/181 mm/hugetlb: use list_splice to merge two list at once
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/hugetlb: use list_splice to merge two list at once
Instead of add allocated file_region one by one to region_cache, we could use list_splice to merge two list at once.
Also we know the number of entries in the list, increase the number directly.
---
mm/hugetlb.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
mm/hugetlb.c renamed from mm/hugetlb.c~mm-hugetlb-use-list_splice-to-merge-two-list-at-once +2 -5
@@ -443,11 +443,8 @@ static int allocate_file_region_entries(spin_lock (&resv->lock); -list_for_each_entry_safe (rg, trg, &allocated_regions, link) { -list_del (&rg->link); -list_add (&rg->link, &resv->region_cache); - resv->region_cache_count++; - } +list_splice (&allocated_regions, &resv->region_cache); + resv->region_cache_count += to_allocate; }return 0 ;
_
Andrew Morton
maintainerPATCH 147/181 mm/hugetlb: count file_region to be added when regions_needed != NULL
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/hugetlb: count file_region to be added when regions_needed != NULL
There are only two cases of function add_reservation_in_range()
* count file_region and return the number in regions_needed
* do the real list operation without counting
This means it is not necessary to have two parameters to classify these two cases.
Just use regions_needed to separate them.
---
mm/hugetlb.c | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-)
mm/hugetlb.c renamed from mm/hugetlb.c~mm-hugetlb-count-file_region-to-be-added-when-regions_needed-=-null +17 -16
@@ -321,16 +321,17 @@ static void coalesce_file_region(struct } } -/* Must be called with resv->lock held. Calling this with count_only == true -* will count the number of pages to be added but will not modify the linked -* list. If regions_needed != NULL and count_only == true, then regions_needed -* will indicate the number of file_regions needed in the cache to carry out to -* add the regions for this range. +/* +* Must be called with resv->lock held. +* +* Calling this with regions_needed != NULL will count the number of pages +* to be added but will not modify the linked list. And regions_needed will +* indicate the number of file_regions needed in the cache to carry out to add +* the regions for this range. */ static long add_reservation_in_range (struct resv_map *resv,long f,long t,struct hugetlb_cgroup *h_cg, -struct hstate *h,long *regions_needed, -bool count_only) +struct hstate *h,long *regions_needed) {long add =0 ;struct list_head *head = &resv->regions;
@@ -366,14 +367,14 @@ static long add_reservation_in_range(str*/ if (rg->from > last_accounted_offset) { add += rg->from - last_accounted_offset; -if (!count_only) { +if (!regions_needed) { nrg =get_file_region_entry_from_cache ( resv, last_accounted_offset, rg->from);record_hugetlb_cgroup_uncharge_info (h_cg, h, resv, nrg);list_add (&nrg->link, rg->link.prev);coalesce_file_region (resv, nrg); - }else if (regions_needed) + }else *regions_needed +=1 ; }
@@ -385,13 +386,13 @@ static long add_reservation_in_range(str*/ if (last_accounted_offset < t) { add += t - last_accounted_offset; -if (!count_only) { +if (!regions_needed) { nrg =get_file_region_entry_from_cache ( resv, last_accounted_offset, t);record_hugetlb_cgroup_uncharge_info (h_cg, h, resv, nrg);list_add (&nrg->link, rg->link.prev);coalesce_file_region (resv, nrg); - }else if (regions_needed) + }else *regions_needed +=1 ; }
@@ -484,8 +485,8 @@ static long region_add(struct resv_map * retry:/* Count how many regions are actually needed to execute this add. */ -add_reservation_in_range (resv, f, t,NULL ,NULL , &actual_regions_needed, - true); +add_reservation_in_range (resv, f, t,NULL ,NULL , + &actual_regions_needed);/* * Check for sufficient descriptors in the cache to accommodate
@@ -513,7 +514,7 @@ retry: goto retry; } - add =add_reservation_in_range (resv, f, t, h_cg, h,NULL , false); + add =add_reservation_in_range (resv, f, t, h_cg, h,NULL ); resv->adds_in_progress -= in_regions_needed;
@@ -549,9 +550,9 @@ static long region_chg(struct resv_map *spin_lock (&resv->lock); -/* Count how many hugepages in this range are NOT respresented. */ +/* Count how many hugepages in this range are NOT represented. */ chg =add_reservation_in_range (resv, f, t,NULL ,NULL , - out_regions_needed, true); + out_regions_needed);if (*out_regions_needed ==0 ) *out_regions_needed =1 ;
_
Andrew Morton
maintainerPATCH 148/181 mm/hugetlb: a page from buddy is not on any list
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/hugetlb: a page from buddy is not on any list
The page allocated from buddy is not on any list, so just use list_add() is enough.
---
mm/hugetlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/hugetlb.c renamed from mm/hugetlb.c~mm-hugetlb-a-page-from-buddy-is-not-on-any-list +1 -1
@@ -2416,7 +2416,7 @@ struct page *alloc_huge_page(struct vm_a h->resv_huge_pages--; }spin_lock (&hugetlb_lock); -list_move (&page->lru, &h->hugepage_activelist); +list_add (&page->lru, &h->hugepage_activelist);/* Fall through */ }hugetlb_cgroup_commit_charge (idx,pages_per_huge_page (h), h_cg, page);
_
Andrew Morton
maintainerPATCH 149/181 mm/hugetlb: narrow the hugetlb_lock protection area during preparing huge page
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/hugetlb: narrow the hugetlb_lock protection area during preparing huge page
set_hugetlb_cgroup_[rsvd] just manipulate page local data, which is not necessary to be protected by hugetlb_lock.
Let's take this out.
---
mm/hugetlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/hugetlb.c renamed from mm/hugetlb.c~mm-hugetlb-narrow-the-hugetlb_lock-protection-area-during-preparing-huge-page +1 -1
@@ -1504,9 +1504,9 @@ static void prep_new_huge_page(struct hs {INIT_LIST_HEAD (&page->lru);set_compound_page_dtor (page,HUGETLB_PAGE_DTOR ); -spin_lock (&hugetlb_lock);set_hugetlb_cgroup (page,NULL );set_hugetlb_cgroup_rsvd (page,NULL ); +spin_lock (&hugetlb_lock); h->nr_huge_pages++; h->nr_huge_pages_node[nid]++;spin_unlock (&hugetlb_lock);
_
Andrew Morton
maintainerPATCH 150/181 mm/hugetlb: take the free hpage during the iteration directly
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/hugetlb: take the free hpage during the iteration directly
Function dequeue_huge_page_node_exact() iterates the free list and return the first valid free hpage.
Instead of break and check the loop variant, we could return in the loop directly. This could reduce some redundant check.
---
mm/hugetlb.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-)
mm/hugetlb.c renamed from mm/hugetlb.c~mm-hugetlb-take-the-free-hpage-during-the-iteration-directly +9 -13
@@ -1040,21 +1040,17 @@ static struct page *dequeue_huge_page_noif (nocma &&is_migrate_cma_page (page))continue ; -if (!PageHWPoison (page)) -break ; +if (PageHWPoison (page)) +continue ; + +list_move (&page->lru, &h->hugepage_activelist); +set_page_refcounted (page); + h->free_huge_pages--; + h->free_huge_pages_node[nid]--; +return page; } -/* -* if 'non-isolated free hugepage' not found on the list, -* the allocation fails. -*/ -if (&h->hugepage_freelists[nid] == &page->lru) -return NULL ; -list_move (&page->lru, &h->hugepage_activelist); -set_page_refcounted (page); - h->free_huge_pages--; - h->free_huge_pages_node[nid]--; -return page; +return NULL ; } static struct page *dequeue_huge_page_nodemask(struct hstate *h, gfp_t gfp_mask, int nid,
_
Andrew Morton
maintainerPATCH 151/181 hugetlb: add lockdep check for i_mmap_rwsem held in huge_pmd_share
In reply to Andrew MortonFrom: Mike Kravetz <[email protected]>
Subject: hugetlb: add lockdep check for i_mmap_rwsem held in huge_pmd_share
As a debugging aid, huge_pmd_share should make sure i_mmap_rwsem is held if necessary. To clarify the 'if necessary', expand the comment block at the beginning of huge_pmd_share.
No functional change. The added i_mmap_assert_locked() call is only enabled if CONFIG_LOCKDEP.
Ideally, this should have been included with commit 34ae204f1851 ("hugetlbfs: remove call to huge_pte_alloc without i_mmap_rwsem").
---
mm/hugetlb.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
mm/hugetlb.c renamed from mm/hugetlb.c~hugetlb-add-lockdep-check-for-i_mmap_rwsem-held-in-huge_pmd_share +11 -4
@@ -5337,10 +5337,16 @@ void adjust_range_if_pmd_sharing_possibl* !shared pmd case because we can allocate the pmd later as well, it makes the * code much cleaner. * -* This routine must be called with i_mmap_rwsem held in at least read mode. -* For hugetlbfs, this prevents removal of any page table entries associated -* with the address space. This is important as we are setting up sharing -* based on existing page table entries (mappings). +* This routine must be called with i_mmap_rwsem held in at least read mode if +* sharing is possible. For hugetlbfs, this prevents removal of any page +* table entries associated with the address space. This is important as we +* are setting up sharing based on existing page table entries (mappings). +* +* NOTE: This routine is only called from huge_pte_alloc. Some callers of +* huge_pte_alloc know that sharing is not possible and do not take +* i_mmap_rwsem as a performance optimization. This is handled by the +* if !vma_shareable check at the beginning of the routine. i_mmap_rwsem is +* only required for subsequent processing. */ pte_t *huge_pmd_share (struct mm_struct *mm,unsigned long addr,pud_t *pud) {
@@ -5357,6 +5363,7 @@ pte_t *huge_pmd_share(struct mm_struct *if (!vma_shareable (vma, addr))return (pte_t *)pmd_alloc (mm, pud, addr); +i_mmap_assert_locked (mapping); vma_interval_tree_foreach(svma, &mapping->i_mmap, idx, idx) {if (svma == vma)continue ;
_
Andrew Morton
maintainerPATCH 152/181 mm/vmscan: fix infinite loop in drop_slab_node
In reply to Andrew MortonFrom: Chunxin Zang <[email protected]>
Subject: mm/vmscan: fix infinite loop in drop_slab_node
We have observed that drop_caches can take a considerable amount of time (<put data here>). Especially when there are many memcgs involved because they are adding an additional overhead.
It is quite unfortunate that the operation cannot be interrupted by a signal currently. Add a check for fatal signals into the main loop so that userspace can control early bailout.
There are two reasons:
1. We have too many memcgs, even though one object freed in one memcg, the sum of object is bigger than 10.
2. We spend a lot of time in traverse memcg once. So, the memcg who traversed at the first have been freed many objects. Traverse memcg next time, the freed count bigger than 10 again.
We can get the following info through 'ps':
root:~# ps -aux | grep drop root 357956 ... R Aug25 21119854:55 echo 3 > /proc/sys/vm/drop_caches root 1771385 ... R Aug16 21146421:17 echo 3 > /proc/sys/vm/drop_caches root 1986319 ... R 18:56 117:27 echo 3 > /proc/sys/vm/drop_caches root 2002148 ... R Aug24 5720:39 echo 3 > /proc/sys/vm/drop_caches root 2564666 ... R 18:59 113:58 echo 3 > /proc/sys/vm/drop_caches root 2639347 ... R Sep03 2383:39 echo 3 > /proc/sys/vm/drop_caches root 3904747 ... R 03:35 993:31 echo 3 > /proc/sys/vm/drop_caches root 4016780 ... R Aug21 7882:18 echo 3 > /proc/sys/vm/drop_caches
Use bpftrace follow 'freed' value in drop_slab_node:
root:~# bpftrace -e 'kprobe:drop_slab_node+70 {@ret=hist(reg("bp")); }'
Attaching 1 probe...
^B^C@ret:
[64, 128) 1 | | [128, 256) 28 | | [256, 512) 107 |@ | [512, 1K) 298 |@@@ | [1K, 2K) 613 |@@@@@@@ | [2K, 4K) 4435 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [4K, 8K) 442 |@@@@@ | [8K, 16K) 299 |@@@ | [16K, 32K) 100 |@ | [32K, 64K) 139 |@ | [64K, 128K) 56 | | [128K, 256K) 26 | | [256K, 512K) 2 | |
In the while loop, we can check whether the TASK_KILLABLE signal is set, if so, we should break the loop.
---
mm/vmscan.c | 3 +++ 1 file changed, 3 insertions(+)
mm/vmscan.c renamed from mm/vmscan.c~mm-vmscan-fix-infinite-loop-in-drop_slab_node +3 -0
@@ -699,6 +699,9 @@ void drop_slab_node(int nid) do { struct mem_cgroup *memcg = NULL; + if (fatal_signal_pending(current)) + return; + freed = 0; memcg = mem_cgroup_iter(NULL, NULL, NULL); do {
_
Andrew Morton
maintainerPATCH 153/181 mm/vmscan: fix comments for isolate_lru_page()
In reply to Andrew MortonFrom: Hui Su <[email protected]>
Subject: mm/vmscan: fix comments for isolate_lru_page()
fix comments for isolate_lru_page():
s/fundamentnal/fundamental
---
mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmscan.c renamed from mm/vmscan.c~mm-vmscan-fix-comments-for-isolate_lru_page +1 -1
@@ -1754,7 +1754,7 @@ static unsigned long isolate_lru_pages(u* Restrictions: * * (1) Must be called with an elevated refcount on the page. This is a -* fundamentnal difference from isolate_lru_pages (which is called +* fundamental difference from isolate_lru_pages (which is called * without a stable reference). * (2) the lru_lock must not be held. * (3) interrupts must be enabled.
_
Andrew Morton
maintainerPATCH 154/181 mm/z3fold.c: use xx_zalloc instead xx_alloc and memset
In reply to Andrew MortonFrom: Hui Su <[email protected]>
Subject: mm/z3fold.c: use xx_zalloc instead xx_alloc and memset
alloc_slots() allocates memory for slots using kmem_cache_alloc(), then memsets it. We can just use kmem_cache_zalloc().
---
mm/z3fold.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
mm/z3fold.c renamed from mm/z3fold.c~mmz3fold-use-xx_zalloc-instead-xx_alloc-and-memset +1 -2
@@ -212,13 +212,12 @@ static inline struct z3fold_buddy_slots {struct z3fold_buddy_slots *slots; - slots =kmem_cache_alloc (pool->c_handle, + slots =kmem_cache_zalloc (pool->c_handle, (gfp & ~(__GFP_HIGHMEM | __GFP_MOVABLE)));if (slots) {/* It will be freed separately in free_handle(). */ kmemleak_not_leak (slots); -memset (slots->slot,0 ,sizeof (slots->slot)); slots->pool = (unsigned long )pool;rwlock_init (&slots->lock); }
_
Andrew Morton
maintainerPATCH 155/181 mm/zbud: remove redundant initialization
In reply to Andrew MortonFrom: Xiang Chen <[email protected]>
Subject: mm/zbud: remove redundant initialization
zhdr is already initialized in the front of the function, so remove redundant initialization here.
---
mm/zbud.c | 1 - 1 file changed, 1 deletion(-)
mm/zbud.c renamed from mm/zbud.c~mm-zbud-remove-redundant-initialization +0 -1
@@ -367,7 +367,6 @@ int zbud_alloc(struct zbud_pool *pool, sspin_lock (&pool->lock);/* First, try to find an unbuddied zbud page. */ - zhdr =NULL ;for_each_unbuddied_list (i, chunks) {if (!list_empty (&pool->unbuddied[i])) { zhdr = list_first_entry(&pool->unbuddied[i],
_
Andrew Morton
maintainerPATCH 156/181 mm/compaction.c: micro-optimization remove unnecessary branch
In reply to Andrew MortonFrom: Mateusz Nosek <[email protected]>
Subject: mm/compaction.c: micro-optimization remove unnecessary branch
The same code can work both for 'zone->compact_considered > defer_limit' and 'zone->compact_considered >= defer_limit'. In the latter there is one branch less which is more effective considering performance.
---
mm/compaction.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
mm/compaction.c renamed from mm/compaction.c~mm-compactionc-micro-optimization-remove-unnecessary-branch +2 -3
@@ -180,11 +180,10 @@ bool compaction_deferred(struct zone *zoreturn false;/* Avoid possible overflow */ -if (++zone->compact_considered > defer_limit) +if (++zone->compact_considered >= defer_limit) { zone->compact_considered = defer_limit; - -if (zone->compact_considered >= defer_limit)return false; + }trace_mm_compaction_deferred (zone, order);
_
Andrew Morton
maintainerPATCH 157/181 include/linux/compaction.h: clean code by removing unused enum value
In reply to Andrew MortonFrom: Mateusz Nosek <[email protected]>
Subject: include/linux/compaction.h: clean code by removing unused enum value
The enum value 'COMPACT_INACTIVE' is never used so can be removed.
---
include/linux/compaction.h | 3 --- 1 file changed, 3 deletions(-)
include/linux/compaction.h renamed from include/linux/compaction.h~include-linux-compactionh-clean-code-by-removing-unused-enum-value +0 -3
@@ -29,9 +29,6 @@ enum compact_result {/* compaction didn't start as it was deferred due to past failures */ COMPACT_DEFERRED , -/* compaction not active last round */ -COMPACT_INACTIVE =COMPACT_DEFERRED , -/* For more detailed tracepoint output - internal to compaction */ COMPACT_NO_SUITABLE_PAGE,/* compaction should continue to another pageblock */
_
Andrew Morton
maintainerPATCH 158/181 selftests/vm: 8x compaction_test speedup
In reply to Andrew MortonFrom: John Hubbard <[email protected]>
Subject: selftests/vm: 8x compaction_test speedup
This patch reduces the running time for compaction_test from about 27 sec, to 3.3 sec, which is about an 8x speedup.
These numbers are for an Intel x86_64 system with 32 GB of DRAM.
The compaction_test.c program was spending most of its time doing mmap(), 1 MB at a time, on about 25 GB of memory.
Instead, do the mmaps 100 MB at a time. (Going past 100 MB doesn't make things go much faster, because other parts of the program are using the remaining time.)
---
tools/testing/selftests/vm/compaction_test.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
tools/testing/selftests/vm/compaction_test.c renamed from tools/testing/selftests/vm/compaction_test.c~selftests-vm-8x-compaction_test-speedup +6 -5
@@ -18,7 +18,8 @@#include "../kselftest.h" -#define MAP_SIZE 1048576 +#define MAP_SIZE_MB 100 +#define MAP_SIZE (MAP_SIZE_MB * 1024 * 1024)struct map_list {void *map;
@@ -165,7 +166,7 @@ int main(int argc, char **argv)void *map =NULL ;unsigned long mem_free =0 ;unsigned long hugepage_size =0 ; -unsigned long mem_fragmentable =0 ; +long mem_fragmentable_MB =0 ;if (prereq () !=0 ) { printf("Either the sysctl compact_unevictable_allowed is not\n"
@@ -190,9 +191,9 @@ int main(int argc, char **argv)return -1 ; } - mem_fragmentable = mem_free *0.8 /1024 ; + mem_fragmentable_MB = mem_free *0.8 /1024 ; -while (mem_fragmentable >0 ) { +while (mem_fragmentable_MB >0 ) { map =mmap (NULL ,MAP_SIZE ,PROT_READ |PROT_WRITE ,MAP_ANONYMOUS |MAP_PRIVATE |MAP_LOCKED ,-1 ,0 );if (map ==MAP_FAILED )
@@ -213,7 +214,7 @@ int main(int argc, char **argv)for (i =0 ; i <MAP_SIZE ; i += page_size) *(unsigned long *)(map + i) = (unsigned long )map + i; - mem_fragmentable--; + mem_fragmentable_MB -=MAP_SIZE_MB ; }for (entry = list; entry !=NULL ; entry = entry->next) {
_
Andrew Morton
maintainerPATCH 159/181 mm/mempolicy: remove or narrow the lock on current
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm/mempolicy: remove or narrow the lock on current
It is not necessary to hold the lock of current when setting nodemask of a new policy.
---
mm/mempolicy.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
mm/mempolicy.c renamed from mm/mempolicy.c~mm-mempolicy-remove-or-narrow-the-lock-on-current +1 -4
@@ -875,13 +875,12 @@ static long do_set_mempolicy(unsigned sh goto out; } -task_lock (current); ret =mpol_set_nodemask (new, nodes, scratch);if (ret) { -task_unlock (current);mpol_put (new); goto out; } +task_lock (current); old = current->mempolicy; current->mempolicy = new;if (new && new->mode ==MPOL_INTERLEAVE )
@@ -1324,9 +1323,7 @@ static long do_mbind(unsigned long startNODEMASK_SCRATCH (scratch);if (scratch) {mmap_write_lock (mm); -task_lock (current); err =mpol_set_nodemask (new, nmask, scratch); -task_unlock (current);if (err)mmap_write_unlock (mm); }else
_
Andrew Morton
maintainerPATCH 160/181 mm: remove unused alloc_page_vma_node()
In reply to Andrew MortonFrom: Wei Yang <[email protected]>
Subject: mm: remove unused alloc_page_vma_node()
No one use this macro anymore.
Also fix code style of policy_node().
---
include/linux/gfp.h | 2 -- mm/mempolicy.c | 3 +-- 2 files changed, 1 insertion(+), 4 deletions(-)
include/linux/gfp.h renamed from include/linux/gfp.h~mm-remove-unused-alloc_page_vma_node +0 -2
@@ -562,8 +562,6 @@ extern struct page *alloc_pages_vma(gfp_#define alloc_page (gfp_mask) alloc_pages(gfp_mask, 0)#define alloc_page_vma (gfp_mask, vma, addr) \ alloc_pages_vma(gfp_mask, 0, vma, addr, numa_node_id(), false) -#define alloc_page_vma_node (gfp_mask, vma, addr, node) \ - alloc_pages_vma(gfp_mask, 0, vma, addr, node, false)extern unsigned long __get_free_pages (gfp_t gfp_mask,unsigned int order);extern unsigned long get_zeroed_page (gfp_t gfp_mask);
mm/mempolicy.c renamed from mm/mempolicy.c~mm-remove-unused-alloc_page_vma_node +1 -2
@@ -1882,8 +1882,7 @@ nodemask_t *policy_nodemask(gfp_t gfp, s }/* Return the node id preferred by the given mempolicy, or the given id */ -static int policy_node (gfp_t gfp,struct mempolicy *policy, -int nd) +static int policy_node (gfp_t gfp,struct mempolicy *policy,int nd) {if (policy->mode ==MPOL_PREFERRED && !(policy->flags &MPOL_F_LOCAL )) nd = policy->v.preferred_node;
_
Andrew Morton
maintainerPATCH 161/181 mm/mempool: add 'else' to split mutually exclusive case
In reply to Andrew MortonFrom: Miaohe Lin <[email protected]>
Subject: mm/mempool: add 'else' to split mutually exclusive case
Add else to split mutually exclusive case and avoid some unnecessary check. It doesn't seem to change code generation (compiler is smart), but I think it helps readability.
---
mm/mempool.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
mm/mempool.c renamed from mm/mempool.c~mm-mempool-add-else-to-split-mutually-exclusive-case +8 -10
@@ -58,11 +58,10 @@ static void __check_element(mempool_t *pstatic void check_element (mempool_t *pool,void *element) {/* Mempools backed by slab allocator */ -if (pool->free == mempool_free_slab || pool->free == mempool_kfree) +if (pool->free == mempool_free_slab || pool->free == mempool_kfree) {__check_element (pool, element,ksize (element)); - -/* Mempools backed by page allocator */ -if (pool->free == mempool_free_pages) { + }else if (pool->free == mempool_free_pages) { +/* Mempools backed by page allocator */ int order = (int )(long )pool->pool_data;void *addr =kmap_atomic ((struct page *)element);
@@ -82,11 +81,10 @@ static void __poison_element(void *elemestatic void poison_element (mempool_t *pool,void *element) {/* Mempools backed by slab allocator */ -if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc) +if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc) {__poison_element (element,ksize (element)); - -/* Mempools backed by page allocator */ -if (pool->alloc == mempool_alloc_pages) { + }else if (pool->alloc == mempool_alloc_pages) { +/* Mempools backed by page allocator */ int order = (int )(long )pool->pool_data;void *addr =kmap_atomic ((struct page *)element);
@@ -107,7 +105,7 @@ static __always_inline void kasan_poison {if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc)kasan_poison_kfree (element, _RET_IP_); -if (pool->alloc == mempool_alloc_pages) +else if (pool->alloc == mempool_alloc_pages)kasan_free_pages (element, (unsigned long )pool->pool_data); }
@@ -115,7 +113,7 @@ static void kasan_unpoison_element(mempo {if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc)kasan_unpoison_slab (element); -if (pool->alloc == mempool_alloc_pages) +else if (pool->alloc == mempool_alloc_pages)kasan_alloc_pages (element, (unsigned long )pool->pool_data); }
_
Andrew Morton
maintainerPATCH 162/181 KVM: PPC: Book3S HV: simplify kvm_cma_reserve()
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: KVM: PPC: Book3S HV: simplify kvm_cma_reserve()
Patch series "memblock: seasonal cleaning^w cleanup", v3.
These patches simplify several uses of memblock iterators and hide some of the memblock implementation details from the rest of the system.
This patch (of 17):
The memory size calculation in kvm_cma_reserve() traverses memblock.memory rather than simply call memblock_phys_mem_size(). The comment in that function suggests that at some point there should have been call to memblock_analyze() before memblock_phys_mem_size() could be used. As of now, there is no memblock_analyze() at all and memblock_phys_mem_size() can be used as soon as cold-plug memory is registered with memblock.
Replace loop over memblock.memory with a call to memblock_phys_mem_size().
---
arch/powerpc/kvm/book3s_hv_builtin.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-)
arch/powerpc/kvm/book3s_hv_builtin.c renamed from arch/powerpc/kvm/book3s_hv_builtin.c~kvm-ppc-book3s-hv-simplify-kvm_cma_reserve +2 -10
@@ -95,23 +95,15 @@ EXPORT_SYMBOL_GPL(kvm_free_hpt_cma);void __init kvm_cma_reserve (void ) {unsigned long align_size; -struct memblock_region *reg; -phys_addr_t selected_size =0 ; +phys_addr_t selected_size;/* * We need CMA reservation only when we are in HV mode */ if (!cpu_has_feature (CPU_FTR_HVMODE ))return ; -/* -* We cannot use memblock_phys_mem_size() here, because -* memblock_analyze() has not been called yet. -*/ -for_each_memblock (memory, reg) - selected_size +=memblock_region_memory_end_pfn (reg) - -memblock_region_memory_base_pfn (reg); - selected_size = (selected_size * kvm_cma_resv_ratio /100 ) <<PAGE_SHIFT ; + selected_size =PAGE_ALIGN (memblock_phys_mem_size () * kvm_cma_resv_ratio /100 );if (selected_size) {pr_info ("%s: reserving %ld MiB for global area\n" , __func__, (unsigned long )selected_size /SZ_1M );
_
Andrew Morton
maintainerPATCH 163/181 dma-contiguous: simplify cma_early_percent_memory()
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: dma-contiguous: simplify cma_early_percent_memory()
The memory size calculation in cma_early_percent_memory() traverses memblock.memory rather than simply call memblock_phys_mem_size(). The comment in that function suggests that at some point there should have been call to memblock_analyze() before memblock_phys_mem_size() could be used. As of now, there is no memblock_analyze() at all and memblock_phys_mem_size() can be used as soon as cold-plug memory is registered with memblock.
Replace loop over memblock.memory with a call to memblock_phys_mem_size().
---
kernel/dma/contiguous.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-)
kernel/dma/contiguous.c renamed from kernel/dma/contiguous.c~dma-contiguous-simplify-cma_early_percent_memory +1 -10
@@ -73,16 +73,7 @@ early_param("cma", early_cma);static phys_addr_t __init __maybe_unused cma_early_percent_memory (void ) { -struct memblock_region *reg; -unsigned long total_pages =0 ; - -/* -* We cannot use memblock_phys_mem_size() here, because -* memblock_analyze() has not been called yet. -*/ -for_each_memblock (memory, reg) - total_pages +=memblock_region_memory_end_pfn (reg) - -memblock_region_memory_base_pfn (reg); +unsigned long total_pages =PHYS_PFN (memblock_phys_mem_size ());return (total_pages *CONFIG_CMA_SIZE_PERCENTAGE /100 ) <<PAGE_SHIFT ; }
_
Andrew Morton
maintainerPATCH 164/181 arm, xtensa: simplify initialization of high memory pages
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: arm, xtensa: simplify initialization of high memory pages
free_highpages() in both arm and xtensa essentially open-code for_each_free_mem_range() loop to detect high memory pages that were not reserved and that should be initialized and passed to the buddy allocator.
Replace open-coded implementation of for_each_free_mem_range() with usage of memblock API to simplify the code.
---
arch/arm/mm/init.c | 48 +++++----------------------------- arch/xtensa/mm/init.c | 55 +++++++--------------------------------- 2 files changed, 18 insertions(+), 85 deletions(-)
arch/arm/mm/init.c renamed from arch/arm/mm/init.c~arm-xtensa-simplify-initialization-of-high-memory-pages +8 -40
@@ -347,61 +347,29 @@ static void __init free_unused_memmap(vo#endif } -#ifdef CONFIG_HIGHMEM -static inline void free_area_high (unsigned long pfn,unsigned long end) -{ -for (; pfn < end; pfn++) -free_highmem_page (pfn_to_page (pfn)); -} -#endif -static void __init free_highpages (void ) {#ifdef CONFIG_HIGHMEM unsigned long max_low = max_low_pfn; -struct memblock_region *mem, *res; +phys_addr_t range_start, range_end; +u64 i;/* set highmem page free */ -for_each_memblock (memory, mem) { -unsigned long start =memblock_region_memory_base_pfn (mem); -unsigned long end =memblock_region_memory_end_pfn (mem); +for_each_free_mem_range (i,NUMA_NO_NODE ,MEMBLOCK_NONE , + &range_start, &range_end,NULL ) { +unsigned long start =PHYS_PFN (range_start); +unsigned long end =PHYS_PFN (range_end);/* Ignore complete lowmem entries */ if (end <= max_low)continue ; -if (memblock_is_nomap (mem)) -continue ; -/* Truncate partial highmem entries */ if (start < max_low) start = max_low; -/* Find and exclude any reserved regions */ -for_each_memblock (reserved, res) { -unsigned long res_start, res_end; - - res_start =memblock_region_reserved_base_pfn (res); - res_end =memblock_region_reserved_end_pfn (res); - -if (res_end < start) -continue ; -if (res_start < start) - res_start = start; -if (res_start > end) - res_start = end; -if (res_end > end) - res_end = end; -if (res_start != start) -free_area_high (start, res_start); - start = res_end; -if (start == end) -break ; - } - -/* And now free anything which remains */ -if (start < end) -free_area_high (start, end); +for (; start < end; start++) +free_highmem_page (pfn_to_page (start)); }#endif }
arch/xtensa/mm/init.c renamed from arch/xtensa/mm/init.c~arm-xtensa-simplify-initialization-of-high-memory-pages +10 -45
77 lines not on this page · raw message
_
Andrew Morton
maintainerPATCH 165/181 arm64: numa: simplify dummy_numa_init()
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: arm64: numa: simplify dummy_numa_init()
dummy_numa_init() loops over memblock.memory and passes nid=0 to numa_add_memblk() which essentially wraps memblock_set_node(). However, memblock_set_node() can cope with entire memory span itself, so the loop over memblock.memory regions is redundant.
Using a single call to memblock_set_node() rather than a loop also fixes an issue with a buggy ACPI firmware in which the SRAT table covers some but not all of the memory in the EFI memory map.
Jonathan Cameron says:
This issue can be easily triggered by having an SRAT table which fails to cover all elements of the EFI memory map.
This firmware error is detected and a warning printed. e.g. "NUMA: Warning: invalid memblk node 64 [mem 0x240000000-0x27fffffff]" At that point we fall back to dummy_numa_init().
However, the failed ACPI init has left us with our memblocks all broken up as we split them when trying to assign them to NUMA nodes.
We then iterate over the memblocks and add them to node 0.
numa_add_memblk() calls memblock_set_node() which merges regions that were previously split up during the earlier attempt to add them to different nodes during parsing of SRAT.
This means elements are moved in the memblock array and we can end up in a different memblock after the call to numa_add_memblk(). Result is:
Unable to handle kernel paging request at virtual address 0000000000003a40
Mem abort info:
ESR = 0x96000004
EC = 0x25: DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
Data abort info:
ISV = 0, ISS = 0x00000004
CM = 0, WnR = 0
[0000000000003a40] user address but active_mm is swapper
Internal error: Oops: 96000004 [#1] PREEMPT SMP...
Call trace:
sparse_init_nid+0x5c/0x2b0
sparse_init+0x138/0x170
bootmem_init+0x80/0xe0
setup_arch+0x2a0/0x5fc
start_kernel+0x8c/0x648Replace the loop with a single call to memblock_set_node() to the entire memory.
---
arch/arm64/mm/numa.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-)
arch/arm64/mm/numa.c renamed from arch/arm64/mm/numa.c~arm64-numa-simplify-dummy_numa_init +5 -8
@@ -427,19 +427,16 @@ out_free_distance:*/ static int __init dummy_numa_init (void ) { +phys_addr_t start =memblock_start_of_DRAM (); +phys_addr_t end =memblock_end_of_DRAM ();int ret; -struct memblock_region *mblk;if (numa_off)pr_info ("NUMA disabled\n" );/* Forced off on command line. */ -pr_info ("Faking a node at [mem %#018Lx-%#018Lx]\n" , -memblock_start_of_DRAM (),memblock_end_of_DRAM () -1 ); - -for_each_memblock (memory, mblk) { - ret =numa_add_memblk (0 , mblk->base, mblk->base + mblk->size); -if (!ret) -continue ; +pr_info ("Faking a node at [mem %#018Lx-%#018Lx]\n" , start, end -1 ); + ret =numa_add_memblk (0 , start, end); +if (ret) {pr_err ("NUMA init failed\n" );return ret; }
_
Andrew Morton
maintainerPATCH 166/181 h8300, nds32, openrisc: simplify detection of memory extents
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: h8300, nds32, openrisc: simplify detection of memory extents
Instead of traversing memblock.memory regions to find memory_start and memory_end, simply query memblock_{start,end}_of_DRAM().
---
arch/h8300/kernel/setup.c | 8 +++----- arch/nds32/kernel/setup.c | 8 ++------ arch/openrisc/kernel/setup.c | 9 ++------- 3 files changed, 7 insertions(+), 18 deletions(-)
arch/h8300/kernel/setup.c renamed from arch/h8300/kernel/setup.c~h8300-nds32-openrisc-simplify-detection-of-memory-extents +3 -5
@@ -74,17 +74,15 @@ static void __init bootmem_init(void) memory_end = memory_start =0 ;/* Find main memory where is the kernel */ -for_each_memblock (memory, region) { - memory_start = region->base; - memory_end = region->base + region->size; - } + memory_start =memblock_start_of_DRAM (); + memory_end =memblock_end_of_DRAM ();if (!memory_end)panic ("No memory!" );/* setup bootmem globals (we use no_bootmem, but mm still depends on this) */ min_low_pfn =PFN_UP (memory_start); - max_low_pfn =PFN_DOWN (memblock_end_of_DRAM ()); + max_low_pfn =PFN_DOWN (memory_end); max_pfn = max_low_pfn;memblock_reserve (__pa (_stext), _end - _stext);
arch/nds32/kernel/setup.c renamed from arch/nds32/kernel/setup.c~h8300-nds32-openrisc-simplify-detection-of-memory-extents +2 -6
@@ -249,12 +249,8 @@ static void __init setup_memory(void) memory_end = memory_start =0 ;/* Find main memory where is the kernel */ -for_each_memblock (memory, region) { - memory_start = region->base; - memory_end = region->base + region->size; -pr_info ("%s: Memory: 0x%x-0x%x\n" , __func__, - memory_start, memory_end); - } + memory_start =memblock_start_of_DRAM (); + memory_end =memblock_end_of_DRAM ();if (!memory_end) {panic ("No memory!" );
arch/openrisc/kernel/setup.c renamed from arch/openrisc/kernel/setup.c~h8300-nds32-openrisc-simplify-detection-of-memory-extents +2 -7
@@ -48,17 +48,12 @@ static void __init setup_memory(void)unsigned long ram_start_pfn;unsigned long ram_end_pfn;phys_addr_t memory_start, memory_end; -struct memblock_region *region; memory_end = memory_start =0 ;/* Find main memory where is the kernel, we assume its the only one */ -for_each_memblock (memory, region) { - memory_start = region->base; - memory_end = region->base + region->size; -printk (KERN_INFO "%s: Memory: 0x%x-0x%x\n" , __func__, - memory_start, memory_end); - } + memory_start =memblock_start_of_DRAM (); + memory_end =memblock_end_of_DRAM ();if (!memory_end) {panic ("No memory!" );
_
Andrew Morton
maintainerPATCH 167/181 riscv: drop unneeded node initialization
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: riscv: drop unneeded node initialization
RISC-V does not (yet) support NUMA and for UMA architectures node 0 is used implicitly during early memory initialization.
There is no need to call memblock_set_node(), remove this call and the surrounding code.
---
arch/riscv/mm/init.c | 9 --------- 1 file changed, 9 deletions(-)
arch/riscv/mm/init.c renamed from arch/riscv/mm/init.c~riscv-drop-unneeded-node-initialization +0 -9
@@ -191,15 +191,6 @@ void __init setup_bootmem(void)early_init_fdt_scan_reserved_mem ();memblock_allow_resize ();memblock_dump_all (); - -for_each_memblock (memory, reg) { -unsigned long start_pfn =memblock_region_memory_base_pfn (reg); -unsigned long end_pfn =memblock_region_memory_end_pfn (reg); - -memblock_set_node (PFN_PHYS (start_pfn), -PFN_PHYS (end_pfn - start_pfn), - &memblock.memory,0 ); - } }#ifdef CONFIG_MMU
_
Andrew Morton
maintainerPATCH 168/181 mircoblaze: drop unneeded NUMA and sparsemem initializations
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: mircoblaze: drop unneeded NUMA and sparsemem initializations
microblaze does not support neither NUMA not SPARSMEM, so there is no point to call memblock_set_node() and sparse_memory_present_with_active_regions() functions during microblaze memory initialization.
Remove these calls and the surrounding code.
---
arch/microblaze/mm/init.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-)
arch/microblaze/mm/init.c renamed from arch/microblaze/mm/init.c~mircoblaze-drop-unneeded-numa-and-sparsemem-initializations +1 -13
@@ -108,9 +108,8 @@ static void __init paging_init(void)void __init setup_memory (void ) { -struct memblock_region *reg; -#ifndef CONFIG_MMU +struct memblock_region *reg;u32 kernel_align_start, kernel_align_size;/* Find main memory where is the kernel */
@@ -164,17 +163,6 @@ void __init setup_memory(void)pr_info ("%s: max_low_pfn: %#lx\n" , __func__, max_low_pfn);pr_info ("%s: max_pfn: %#lx\n" , __func__, max_pfn); -/* Add active regions with valid PFNs */ -for_each_memblock (memory, reg) { -unsigned long start_pfn, end_pfn; - - start_pfn =memblock_region_memory_base_pfn (reg); - end_pfn =memblock_region_memory_end_pfn (reg); -memblock_set_node (start_pfn <<PAGE_SHIFT , - (end_pfn - start_pfn) <<PAGE_SHIFT , - &memblock.memory,0 ); - } -paging_init (); }
_
Andrew Morton
maintainerPATCH 169/181 memblock: make for_each_memblock_type() iterator private
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: memblock: make for_each_memblock_type() iterator private
for_each_memblock_type() is not used outside mm/memblock.c, move it there from include/linux/memblock.h
---
include/linux/memblock.h | 5 ----- mm/memblock.c | 5 +++++ 2 files changed, 5 insertions(+), 5 deletions(-)
include/linux/memblock.h renamed from include/linux/memblock.h~memblock-make-for_each_memblock_type-iterator-private +0 -5
@@ -552,11 +552,6 @@ static inline unsigned long memblock_reg region < (memblock.memblock_type.regions + memblock.memblock_type.cnt); \ region++) -#define for_each_memblock_type (i, memblock_type, rgn) \ - for (i = 0, rgn = &memblock_type->regions[0]; \ - i < memblock_type->cnt; \ - i++, rgn = &memblock_type->regions[i]) - extern void *alloc_large_system_hash(const char *tablename, unsigned long bucketsize, unsigned long numentries,
mm/memblock.c renamed from mm/memblock.c~memblock-make-for_each_memblock_type-iterator-private +5 -0
@@ -132,6 +132,11 @@ struct memblock_type physmem = { };#endif +#define for_each_memblock_type (i, memblock_type, rgn) \ + for (i = 0, rgn = &memblock_type->regions[0]; \ + i < memblock_type->cnt; \ + i++, rgn = &memblock_type->regions[i]) + int memblock_debug __initdata_memblock; static bool system_has_some_mirror __initdata_memblock = false; static int memblock_can_resize __initdata_memblock;
_
Andrew Morton
maintainerPATCH 170/181 memblock: make memblock_debug and related functionality private
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: memblock: make memblock_debug and related functionality private
The only user of memblock_dbg() outside memblock was s390 setup code and it is converted to use pr_debug() instead. This allows to stop exposing memblock_debug and memblock_dbg() to the rest of the kernel.
---
arch/s390/kernel/setup.c | 4 ++-- include/linux/memblock.h | 12 +----------- mm/memblock.c | 16 ++++++++++++++-- 3 files changed, 17 insertions(+), 15 deletions(-)
arch/s390/kernel/setup.c renamed from arch/s390/kernel/setup.c~memblock-make-memblock_debug-and-related-functionality-private +2 -2
@@ -776,8 +776,8 @@ static void __init memblock_add_mem_deteunsigned long start, end;int i; -memblock_dbg ("physmem info source: %s (%hhd)\n" , -get_mem_info_source (), mem_detect.info_source); +pr_debug ("physmem info source: %s (%hhd)\n" , +get_mem_info_source (), mem_detect.info_source);/* keep memblock lists close to the kernel */ memblock_set_bottom_up (true);for_each_mem_detect_block (i, &start, &end) {
include/linux/memblock.h renamed from include/linux/memblock.h~memblock-make-memblock_debug-and-related-functionality-private +1 -11
@@ -86,7 +86,6 @@ struct memblock { };extern struct memblock memblock; -extern int memblock_debug;#ifndef CONFIG_ARCH_KEEP_MEMBLOCK #define __init_memblock __meminit
@@ -98,9 +97,6 @@ void memblock_discard(void);static inline void memblock_discard (void ) {}#endif -#define memblock_dbg (fmt, ...) \ - if (memblock_debug) printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__) -phys_addr_t memblock_find_in_range (phys_addr_t start,phys_addr_t end,phys_addr_t size,phys_addr_t align);void memblock_allow_resize (void );
@@ -476,13 +472,7 @@ bool memblock_is_region_memory(phys_addrbool memblock_is_reserved (phys_addr_t addr);bool memblock_is_region_reserved (phys_addr_t base,phys_addr_t size); -extern void __memblock_dump_all (void ); - -static inline void memblock_dump_all (void ) -{ -if (memblock_debug) -__memblock_dump_all (); -} +void memblock_dump_all (void );/** * memblock_set_current_limit - Set the current allocation limit to allow
mm/memblock.c renamed from mm/memblock.c~memblock-make-memblock_debug-and-related-functionality-private +14 -2
@@ -137,7 +137,13 @@ struct memblock_type physmem = { i < memblock_type->cnt; \ i++, rgn = &memblock_type->regions[i]) -int memblock_debug __initdata_memblock; +#define memblock_dbg (fmt, ...) \ + do { \ + if (memblock_debug) \ + pr_info(fmt, ##__VA_ARGS__); \ + } while (0) + +static int memblock_debug __initdata_memblock; static bool system_has_some_mirror __initdata_memblock = false; static int memblock_can_resize __initdata_memblock; static int memblock_memory_in_slab __initdata_memblock = 0;
@@ -1920,7 +1926,7 @@ static void __init_memblock memblock_dum } } -void __init_memblock __memblock_dump_all(void) +static void __init_memblock __memblock_dump_all(void) {pr_info ("MEMBLOCK configuration:\n" ); pr_info(" memory size = %pa reserved size = %pa\n",
@@ -1934,6 +1940,12 @@ void __init_memblock __memblock_dump_all#endif } +void __init_memblock memblock_dump_all(void) +{ +if (memblock_debug) +__memblock_dump_all (); +} +void __init memblock_allow_resize (void ) { memblock_can_resize =1 ;
_
Andrew Morton
maintainerPATCH 171/181 memblock: reduce number of parameters in for_each_mem_range()
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: memblock: reduce number of parameters in for_each_mem_range()
Currently for_each_mem_range() and for_each_mem_range_rev() iterators are the most generic way to traverse memblock regions. As such, they have 8 parameters and they are hardly convenient to users. Most users choose to utilize one of their wrappers and the only user that actually needs most of the parameters is memblock itself.
To avoid yet another naming for memblock iterators, rename the existing for_each_mem_range[_rev]() to __for_each_mem_range[_rev]() and add a new for_each_mem_range[_rev]() wrappers with only index, start and end parameters.
The new wrapper nicely fits into init_unavailable_mem() and will be used in upcoming changes to simplify memblock traversals.
---
.clang-format | 2 + arch/arm64/kernel/machine_kexec_file.c | 6 +-- arch/powerpc/kexec/file_load_64.c | 6 +-- include/linux/memblock.h | 41 +++++++++++++++++------ mm/page_alloc.c | 3 - 5 files changed, 38 insertions(+), 20 deletions(-)
arch/arm64/kernel/machine_kexec_file.c renamed from arch/arm64/kernel/machine_kexec_file.c~memblock-reduce-number-of-parameters-in-for_each_mem_range +2 -4
@@ -215,8 +215,7 @@ static int prepare_elf_headers(void **adphys_addr_t start, end; nr_ranges =1 ;/* for exclusion of crashkernel region */ -for_each_mem_range (i, &memblock.memory,NULL ,NUMA_NO_NODE , -MEMBLOCK_NONE , &start, &end,NULL ) +for_each_mem_range (i, &start, &end) nr_ranges++; cmem =kmalloc (struct_size (cmem, ranges, nr_ranges),GFP_KERNEL );
@@ -225,8 +224,7 @@ static int prepare_elf_headers(void **ad cmem->max_nr_ranges = nr_ranges; cmem->nr_ranges =0 ; -for_each_mem_range (i, &memblock.memory,NULL ,NUMA_NO_NODE , -MEMBLOCK_NONE , &start, &end,NULL ) { +for_each_mem_range (i, &start, &end) { cmem->ranges[cmem->nr_ranges].start = start; cmem->ranges[cmem->nr_ranges].end = end -1 ; cmem->nr_ranges++;
arch/powerpc/kexec/file_load_64.c renamed from arch/powerpc/kexec/file_load_64.c~memblock-reduce-number-of-parameters-in-for_each_mem_range +2 -4
@@ -250,8 +250,7 @@ static int __locate_mem_hole_top_down(stphys_addr_t start, end;u64 i; -for_each_mem_range_rev (i, &memblock.memory,NULL ,NUMA_NO_NODE , -MEMBLOCK_NONE , &start, &end,NULL ) { +for_each_mem_range_rev (i, &start, &end) {/* * memblock uses [start, end) convention while it is * [start, end] here. Fix the off-by-one to have the
@@ -350,8 +349,7 @@ static int __locate_mem_hole_bottom_up(sphys_addr_t start, end;u64 i; -for_each_mem_range (i, &memblock.memory,NULL ,NUMA_NO_NODE , -MEMBLOCK_NONE , &start, &end,NULL ) { +for_each_mem_range (i, &start, &end) {/* * memblock uses [start, end) convention while it is * [start, end] here. Fix the off-by-one to have the
.clang-format renamed from .clang-format~memblock-reduce-number-of-parameters-in-for_each_mem_range +2 -0
@@ -207,7 +207,9 @@ ForEachMacros: - 'for_each_memblock_type' - 'for_each_memcg_cache_index' - 'for_each_mem_pfn_range' + - '__for_each_mem_range' - 'for_each_mem_range' + - '__for_each_mem_range_rev' - 'for_each_mem_range_rev' - 'for_each_migratetype_order' - 'for_each_msi_entry'
include/linux/memblock.h renamed from include/linux/memblock.h~memblock-reduce-number-of-parameters-in-for_each_mem_range +31 -10
84 lines not on this page · raw message
mm/page_alloc.c renamed from mm/page_alloc.c~memblock-reduce-number-of-parameters-in-for_each_mem_range +1 -2
@@ -6990,8 +6990,7 @@ static void __init init_unavailable_mem(* Loop through unavailable ranges not covered by memblock.memory. */ pgcnt =0 ; -for_each_mem_range (i, &memblock.memory,NULL , -NUMA_NO_NODE ,MEMBLOCK_NONE , &start, &end,NULL ) { +for_each_mem_range (i, &start, &end) {if (next < start) pgcnt +=init_unavailable_range (PFN_DOWN (next),PFN_UP (start));
_
Andrew Morton
maintainerPATCH 172/181 arch, mm: replace for_each_memblock() with for_each_mem_pfn_range()
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: arch, mm: replace for_each_memblock() with for_each_mem_pfn_range()
There are several occurrences of the following pattern:
for_each_memblock(memory, reg) {
start_pfn = memblock_region_memory_base_pfn(reg);
end_pfn = memblock_region_memory_end_pfn(reg);
/* do something with start_pfn and end_pfn */
}Rather than iterate over all memblock.memory regions and each time query for their start and end PFNs, use for_each_mem_pfn_range() iterator to get simpler and clearer code.
---
arch/arm/mm/init.c | 11 ++++------- arch/arm64/mm/init.c | 11 ++++------- arch/powerpc/kernel/fadump.c | 11 ++++++----- arch/powerpc/mm/mem.c | 15 ++++++++------- arch/powerpc/mm/numa.c | 7 ++----- arch/s390/mm/page-states.c | 6 ++---- arch/sh/mm/init.c | 9 +++------ mm/memblock.c | 6 ++---- mm/sparse.c | 10 ++++------ 9 files changed, 35 insertions(+), 51 deletions(-)
arch/arm64/mm/init.c renamed from arch/arm64/mm/init.c~arch-mm-replace-for_each_memblock-with-for_each_mem_pfn_range +4 -7
@@ -471,12 +471,10 @@ static inline void free_memmap(unsigned*/ static void __init free_unused_memmap (void ) { -unsigned long start, prev_end =0 ; -struct memblock_region *reg; - -for_each_memblock (memory, reg) { - start =__phys_to_pfn (reg->base); +unsigned long start, end, prev_end =0 ; +int i; +for_each_mem_pfn_range (i,MAX_NUMNODES , &start, &end,NULL ) {#ifdef CONFIG_SPARSEMEM /* * Take care not to free memmap entries that don't exist due
@@ -496,8 +494,7 @@ static void __init free_unused_memmap(vo* memmap entries are valid from the bank end aligned to * MAX_ORDER_NR_PAGES. */ - prev_end =ALIGN (__phys_to_pfn (reg->base + reg->size), -MAX_ORDER_NR_PAGES ); + prev_end =ALIGN (end,MAX_ORDER_NR_PAGES ); }#ifdef CONFIG_SPARSEMEM
arch/arm/mm/init.c renamed from arch/arm/mm/init.c~arch-mm-replace-for_each_memblock-with-for_each_mem_pfn_range +4 -7
@@ -299,16 +299,14 @@ free_memmap(unsigned long start_pfn, uns*/ static void __init free_unused_memmap (void ) { -unsigned long start, prev_end =0 ; -struct memblock_region *reg; +unsigned long start, end, prev_end =0 ; +int i;/* * This relies on each bank being in address order. * The banks are sorted previously in bootmem_init(). */ -for_each_memblock (memory, reg) { - start =memblock_region_memory_base_pfn (reg); - +for_each_mem_pfn_range (i,MAX_NUMNODES , &start, &end,NULL ) {#ifdef CONFIG_SPARSEMEM /* * Take care not to free memmap entries that don't exist
@@ -336,8 +334,7 @@ static void __init free_unused_memmap(vo* memmap entries are valid from the bank end aligned to * MAX_ORDER_NR_PAGES. */ - prev_end =ALIGN (memblock_region_memory_end_pfn (reg), -MAX_ORDER_NR_PAGES ); + prev_end =ALIGN (end,MAX_ORDER_NR_PAGES ); }#ifdef CONFIG_SPARSEMEM
arch/powerpc/kernel/fadump.c renamed from arch/powerpc/kernel/fadump.c~arch-mm-replace-for_each_memblock-with-for_each_mem_pfn_range +6 -5
@@ -1242,14 +1242,15 @@ static void fadump_free_reserved_memory(*/ static void fadump_release_reserved_area (u64 start,u64 end) { -u64 tstart, tend, spfn, epfn; -struct memblock_region *reg; +u64 tstart, tend, spfn, epfn, reg_spfn, reg_epfn, i; spfn =PHYS_PFN (start); epfn =PHYS_PFN (end); -for_each_memblock (memory, reg) { - tstart =max_t (u64, spfn,memblock_region_memory_base_pfn (reg)); - tend =min_t (u64, epfn,memblock_region_memory_end_pfn (reg)); + +for_each_mem_pfn_range (i,MAX_NUMNODES , ®_spfn, ®_epfn,NULL ) { + tstart =max_t (u64, spfn, reg_spfn); + tend =min_t (u64, epfn, reg_epfn); +if (tstart < tend) {fadump_free_reserved_memory (tstart, tend);
arch/powerpc/mm/mem.c renamed from arch/powerpc/mm/mem.c~arch-mm-replace-for_each_memblock-with-for_each_mem_pfn_range +8 -7
@@ -184,15 +184,16 @@ void __init initmem_init(void)/* mark pages that don't exist as nosave */ static int __init mark_nonram_nosave (void ) { -struct memblock_region *reg, *prev =NULL ; +unsigned long spfn, epfn, prev =0 ; +int i; -for_each_memblock (memory, reg) { -if (prev && -memblock_region_memory_end_pfn (prev) <memblock_region_memory_base_pfn (reg)) -register_nosave_region (memblock_region_memory_end_pfn (prev), -memblock_region_memory_base_pfn (reg)); - prev = reg; +for_each_mem_pfn_range (i,MAX_NUMNODES , &spfn, &epfn,NULL ) { +if (prev && prev < spfn) +register_nosave_region (prev, spfn); + + prev = epfn; } +return 0 ; }#else /* CONFIG_NEED_MULTIPLE_NODES */
arch/powerpc/mm/numa.c renamed from arch/powerpc/mm/numa.c~arch-mm-replace-for_each_memblock-with-for_each_mem_pfn_range +2 -5
@@ -804,17 +804,14 @@ static void __init setup_nonnuma(void)unsigned long total_ram =memblock_phys_mem_size ();unsigned long start_pfn, end_pfn;unsigned int nid =0 ; -struct memblock_region *reg; +int i;printk (KERN_DEBUG "Top of RAM: 0x%lx, Total RAM: 0x%lx\n" , top_of_ram, total_ram);printk (KERN_DEBUG "Memory hole size: %ldMB\n" , (top_of_ram - total_ram) >>20 ); -for_each_memblock (memory, reg) { - start_pfn =memblock_region_memory_base_pfn (reg); - end_pfn =memblock_region_memory_end_pfn (reg); - +for_each_mem_pfn_range (i,MAX_NUMNODES , &start_pfn, &end_pfn,NULL ) {fake_numa_create_new_node (end_pfn, &nid); memblock_set_node(PFN_PHYS(start_pfn), PFN_PHYS(end_pfn - start_pfn),
arch/s390/mm/page-states.c renamed from arch/s390/mm/page-states.c~arch-mm-replace-for_each_memblock-with-for_each_mem_pfn_range +2 -4
@@ -183,9 +183,9 @@ static void mark_kernel_pgd(void)void __init cmma_init_nodat (void ) { -struct memblock_region *reg;struct page *page;unsigned long start, end, ix; +int i;if (cmma_flag <2 )return ;
@@ -193,9 +193,7 @@ void __init cmma_init_nodat(void)mark_kernel_pgd ();/* Set all kernel pages not used for page tables to stable/no-dat */ -for_each_memblock (memory, reg) { - start =memblock_region_memory_base_pfn (reg); - end =memblock_region_memory_end_pfn (reg); +for_each_mem_pfn_range (i,MAX_NUMNODES , &start, &end,NULL ) { page =pfn_to_page (start);for (ix = start; ix < end; ix++, page++) {if (__test_and_clear_bit (PG_arch_1, &page->flags))
arch/sh/mm/init.c renamed from arch/sh/mm/init.c~arch-mm-replace-for_each_memblock-with-for_each_mem_pfn_range +3 -6
@@ -226,15 +226,12 @@ void __init allocate_pgdat(unsigned intstatic void __init do_init_bootmem (void ) { -struct memblock_region *reg; +unsigned long start_pfn, end_pfn; +int i;/* Add active regions with valid PFNs. */ -for_each_memblock (memory, reg) { -unsigned long start_pfn, end_pfn; - start_pfn =memblock_region_memory_base_pfn (reg); - end_pfn =memblock_region_memory_end_pfn (reg); +for_each_mem_pfn_range (i,MAX_NUMNODES , &start_pfn, &end_pfn,NULL )__add_active_range (0 , start_pfn, end_pfn); - }/* All of system RAM sits in node 0 for the non-NUMA case */ allocate_pgdat (0 );
mm/memblock.c renamed from mm/memblock.c~arch-mm-replace-for_each_memblock-with-for_each_mem_pfn_range +2 -4
@@ -1663,12 +1663,10 @@ phys_addr_t __init_memblock memblock_resphys_addr_t __init memblock_mem_size (unsigned long limit_pfn) {unsigned long pages =0 ; -struct memblock_region *r;unsigned long start_pfn, end_pfn; +int i; -for_each_memblock (memory, r) { - start_pfn =memblock_region_memory_base_pfn (r); - end_pfn =memblock_region_memory_end_pfn (r); +for_each_mem_pfn_range (i,MAX_NUMNODES , &start_pfn, &end_pfn,NULL ) { start_pfn = min_t(unsigned long, start_pfn, limit_pfn); end_pfn = min_t(unsigned long, end_pfn, limit_pfn); pages += end_pfn - start_pfn;
mm/sparse.c renamed from mm/sparse.c~arch-mm-replace-for_each_memblock-with-for_each_mem_pfn_range +4 -6
@@ -291,13 +291,11 @@ static void __init memory_present(int ni*/ static void __init memblocks_present (void ) { -struct memblock_region *reg; +unsigned long start, end; +int i, nid; -for_each_memblock (memory, reg) { -memory_present (memblock_get_region_node (reg), -memblock_region_memory_base_pfn (reg), -memblock_region_memory_end_pfn (reg)); - } +for_each_mem_pfn_range (i,MAX_NUMNODES , &start, &end, &nid) +memory_present (nid, start, end); }/*
_
Andrew Morton
maintainerPATCH 173/181 arch, drivers: replace for_each_membock() with for_each_mem_range()
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: arch, drivers: replace for_each_membock() with for_each_mem_range()
There are several occurrences of the following pattern:
for_each_memblock(memory, reg) {
start = __pfn_to_phys(memblock_region_memory_base_pfn(reg);
end = __pfn_to_phys(memblock_region_memory_end_pfn(reg));
/* do something with start and end */
}Using for_each_mem_range() iterator is more appropriate in such cases and allows simpler and cleaner code.
---
arch/arm/kernel/setup.c | 18 +++++-- arch/arm/mm/mmu.c | 39 +++++----------- arch/arm/mm/pmsa-v7.c | 23 ++++----- arch/arm/mm/pmsa-v8.c | 17 +++---- arch/arm/xen/mm.c | 7 +- arch/arm64/mm/kasan_init.c | 10 ++-- arch/arm64/mm/mmu.c | 11 +--- arch/c6x/kernel/setup.c | 9 ++- arch/microblaze/mm/init.c | 9 ++- arch/mips/cavium-octeon/dma-octeon.c | 14 ++--- arch/mips/kernel/setup.c | 31 ++++++------- arch/openrisc/mm/init.c | 8 ++- arch/powerpc/kernel/fadump.c | 50 +++++++++------------ arch/powerpc/kexec/file_load_64.c | 10 +--- arch/powerpc/mm/book3s64/hash_utils.c | 16 +++--- arch/powerpc/mm/book3s64/radix_pgtable.c | 10 ++-- arch/powerpc/mm/kasan/kasan_init_32.c | 8 +-- arch/powerpc/mm/mem.c | 16 ++++-- arch/powerpc/mm/pgtable_32.c | 8 +-- arch/riscv/mm/init.c | 25 ++++------ arch/riscv/mm/kasan_init.c | 10 ++-- arch/s390/kernel/setup.c | 23 ++++++--- arch/s390/mm/vmem.c | 7 +- arch/sparc/mm/init_64.c | 12 +---- drivers/bus/mvebu-mbus.c | 12 ++--- 25 files changed, 195 insertions(+), 208 deletions(-)
arch/arm64/mm/kasan_init.c renamed from arch/arm64/mm/kasan_init.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +5 -5
@@ -212,8 +212,8 @@ void __init kasan_init(void) {u64 kimg_shadow_start, kimg_shadow_end;u64 mod_shadow_start, mod_shadow_end; -struct memblock_region *reg; -int i; +phys_addr_t pa_start, pa_end; +u64 i; kimg_shadow_start = (u64 )kasan_mem_to_shadow (_text) &PAGE_MASK ; kimg_shadow_end =PAGE_ALIGN ((u64 )kasan_mem_to_shadow (_end));
@@ -246,9 +246,9 @@ void __init kasan_init(void)kasan_populate_early_shadow ((void *)mod_shadow_end, (void *)kimg_shadow_start); -for_each_memblock (memory, reg) { -void *start = (void *)__phys_to_virt (reg->base); -void *end = (void *)__phys_to_virt (reg->base + reg->size); +for_each_mem_range (i, &pa_start, &pa_end) { +void *start = (void *)__phys_to_virt (pa_start); +void *end = (void *)__phys_to_virt (pa_end);if (start >= end)break ;
arch/arm64/mm/mmu.c renamed from arch/arm64/mm/mmu.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +3 -8
@@ -473,8 +473,9 @@ static void __init map_mem(pgd_t *pgdp) {phys_addr_t kernel_start =__pa_symbol (_text);phys_addr_t kernel_end =__pa_symbol (__init_begin); -struct memblock_region *reg; +phys_addr_t start, end;int flags =0 ; +u64 i;if (rodata_full ||debug_pagealloc_enabled ()) flags =NO_BLOCK_MAPPINGS |NO_CONT_MAPPINGS ;
@@ -493,15 +494,9 @@ static void __init map_mem(pgd_t *pgdp)#endif /* map all the memory banks */ -for_each_memblock (memory, reg) { -phys_addr_t start = reg->base; -phys_addr_t end = start + reg->size; - +for_each_mem_range (i, &start, &end) {if (start >= end)break ; -if (memblock_is_nomap (reg)) -continue ; -/* * The linear map must allow allocation tags reading/writing * if MTE is present. Otherwise, it has the same attributes as
arch/arm/kernel/setup.c renamed from arch/arm/kernel/setup.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +12 -6
@@ -843,20 +843,26 @@ early_param("mem", early_mem);static void __init request_standard_resources (const struct machine_desc *mdesc) { -struct memblock_region *region; +phys_addr_t start, end, res_end;struct resource *res; +u64 i; kernel_code.start =virt_to_phys (_text); kernel_code.end =virt_to_phys (__init_begin -1 ); kernel_data.start =virt_to_phys (_sdata); kernel_data.end =virt_to_phys (_end -1 ); -for_each_memblock (memory, region) { -phys_addr_t start =__pfn_to_phys (memblock_region_memory_base_pfn (region)); -phys_addr_t end =__pfn_to_phys (memblock_region_memory_end_pfn (region)) -1 ; +for_each_mem_range (i, &start, &end) {unsigned long boot_alias_start;/* +* In memblock, end points to the first byte after the +* range while in resourses, end points to the last byte in +* the range. +*/ + res_end = end -1 ; + +/* * Some systems have a special memory alias which is only * used for booting. We need to advertise this region to * kexec-tools so they know where bootable RAM is located.
@@ -869,7 +875,7 @@ static void __init request_standard_reso __func__, sizeof(*res)); res->name ="System RAM (boot alias)" ; res->start = boot_alias_start; - res->end =phys_to_idmap (end); + res->end =phys_to_idmap (res_end); res->flags =IORESOURCE_MEM |IORESOURCE_BUSY ;request_resource (&iomem_resource, res); }
@@ -880,7 +886,7 @@ static void __init request_standard_reso sizeof(*res)); res->name ="System RAM" ; res->start = start; - res->end = end; + res->end = res_end; res->flags =IORESOURCE_SYSTEM_RAM |IORESOURCE_BUSY ;request_resource (&iomem_resource, res);
arch/arm/mm/mmu.c renamed from arch/arm/mm/mmu.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +13 -26
@@ -1154,9 +1154,8 @@ phys_addr_t arm_lowmem_limit __initdatavoid __init adjust_lowmem_bounds (void ) { -phys_addr_t memblock_limit =0 ; -u64 vmalloc_limit; -struct memblock_region *reg; +phys_addr_t block_start, block_end, memblock_limit =0 ; +u64 vmalloc_limit, i;phys_addr_t lowmem_limit =0 ;/*
@@ -1172,26 +1171,18 @@ void __init adjust_lowmem_bounds(void)* The first usable region must be PMD aligned. Mark its start * as MEMBLOCK_NOMAP if it isn't */ -for_each_memblock (memory, reg) { -if (!memblock_is_nomap (reg)) { -if (!IS_ALIGNED (reg->base,PMD_SIZE )) { -phys_addr_t len; +for_each_mem_range (i, &block_start, &block_end) { +if (!IS_ALIGNED (block_start,PMD_SIZE )) { +phys_addr_t len; - len =round_up (reg->base,PMD_SIZE ) - reg->base; -memblock_mark_nomap (reg->base, len); - } -break ; + len =round_up (block_start,PMD_SIZE ) - block_start; +memblock_mark_nomap (block_start, len); } +break ; } -for_each_memblock (memory, reg) { -phys_addr_t block_start = reg->base; -phys_addr_t block_end = reg->base + reg->size; - -if (memblock_is_nomap (reg)) -continue ; - -if (reg->base < vmalloc_limit) { +for_each_mem_range (i, &block_start, &block_end) { +if (block_start < vmalloc_limit) {if (block_end > lowmem_limit)/* * Compare as u64 to ensure vmalloc_limit does
@@ -1440,19 +1431,15 @@ static void __init kmap_init(void)static void __init map_lowmem (void ) { -struct memblock_region *reg;phys_addr_t kernel_x_start =round_down (__pa (KERNEL_START ),SECTION_SIZE );phys_addr_t kernel_x_end =round_up (__pa (__init_end),SECTION_SIZE ); +phys_addr_t start, end; +u64 i;/* Map all the lowmem memory banks. */ -for_each_memblock (memory, reg) { -phys_addr_t start = reg->base; -phys_addr_t end = start + reg->size; +for_each_mem_range (i, &start, &end) {struct map_desc map; -if (memblock_is_nomap (reg)) -continue ; -if (end > arm_lowmem_limit) end = arm_lowmem_limit;if (start >= end)
arch/arm/mm/pmsa-v7.c renamed from arch/arm/mm/pmsa-v7.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +11 -12
@@ -231,12 +231,12 @@ static int __init allocate_region(phys_avoid __init pmsav7_adjust_lowmem_bounds (void ) {phys_addr_t specified_mem_size =0 , total_mem_size =0 ; -struct memblock_region *reg; -bool first = true;phys_addr_t mem_start;phys_addr_t mem_end; +phys_addr_t reg_start, reg_end;unsigned int mem_max_regions; -int num, i; +int num; +u64 i;/* Free-up PMSAv7_PROBE_REGION */ mpu_min_region_order =__mpu_min_region_order ();
@@ -262,20 +262,19 @@ void __init pmsav7_adjust_lowmem_bounds( mem_max_regions -= num;#endif -for_each_memblock (memory, reg) { -if (first) { +for_each_mem_range (i, ®_start, ®_end) { +if (i ==0 ) {phys_addr_t phys_offset =PHYS_OFFSET ;/* * Initially only use memory continuous from * PHYS_OFFSET */ -if (reg->base != phys_offset) +if (reg_start != phys_offset)panic ("First memory bank must be contiguous from PHYS_OFFSET" ); - mem_start = reg->base; - mem_end = reg->base + reg->size; - specified_mem_size = reg->size; - first = false; + mem_start = reg_start; + mem_end = reg_end; + specified_mem_size = mem_end - mem_start; }else {/* * memblock auto merges contiguous blocks, remove
@@ -283,8 +282,8 @@ void __init pmsav7_adjust_lowmem_bounds(* blocks separately while iterating) */ pr_notice ("Ignoring RAM after %pa, memory at %pa ignored\n" , - &mem_end, ®->base); -memblock_remove (reg->base,0 - reg->base); + &mem_end, ®_start); +memblock_remove (reg_start,0 - reg_start);break ; } }
arch/arm/mm/pmsa-v8.c renamed from arch/arm/mm/pmsa-v8.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +8 -9
@@ -94,20 +94,19 @@ static __init bool is_region_fixed(int nvoid __init pmsav8_adjust_lowmem_bounds (void ) {phys_addr_t mem_end; -struct memblock_region *reg; -bool first = true; +phys_addr_t reg_start, reg_end; +u64 i; -for_each_memblock (memory, reg) { -if (first) { +for_each_mem_range (i, ®_start, ®_end) { +if (i ==0 ) {phys_addr_t phys_offset =PHYS_OFFSET ;/* * Initially only use memory continuous from * PHYS_OFFSET */ -if (reg->base != phys_offset) +if (reg_start != phys_offset)panic ("First memory bank must be contiguous from PHYS_OFFSET" ); - mem_end = reg->base + reg->size; - first = false; + mem_end = reg_end; }else {/* * memblock auto merges contiguous blocks, remove
@@ -115,8 +114,8 @@ void __init pmsav8_adjust_lowmem_bounds(* blocks separately while iterating) */ pr_notice ("Ignoring RAM after %pa, memory at %pa ignored\n" , - &mem_end, ®->base); -memblock_remove (reg->base,0 - reg->base); + &mem_end, ®_start); +memblock_remove (reg_start,0 - reg_start);break ; } }
arch/arm/xen/mm.c renamed from arch/arm/xen/mm.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +4 -3
@@ -25,11 +25,12 @@unsigned long xen_get_swiotlb_free_pages (unsigned int order) { -struct memblock_region *reg; +phys_addr_t base;gfp_t flags = __GFP_NOWARN|__GFP_KSWAPD_RECLAIM; +u64 i; -for_each_memblock (memory, reg) { -if (reg->base < (phys_addr_t )0xffffffff ) { +for_each_mem_range (i, &base,NULL ) { +if (base < (phys_addr_t )0xffffffff ) {if (IS_ENABLED (CONFIG_ZONE_DMA32 )) flags |= __GFP_DMA32;else
arch/c6x/kernel/setup.c renamed from arch/c6x/kernel/setup.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +5 -4
@@ -287,7 +287,8 @@ notrace void __init machine_init(unsignevoid __init setup_arch (char **cmdline_p) { -struct memblock_region *reg; +phys_addr_t start, end; +u64 i;printk (KERN_INFO "Initializing kernel\n" );
@@ -351,9 +352,9 @@ void __init setup_arch(char **cmdline_p)disable_caching (ram_start, ram_end -1 );/* Set caching of external RAM used by Linux */ -for_each_memblock (memory, reg) -enable_caching (CACHE_REGION_START (reg->base), -CACHE_REGION_START (reg->base + reg->size -1 )); +for_each_mem_range (i, &start, &end) +enable_caching (CACHE_REGION_START (start), +CACHE_REGION_START (end -1 ));#ifdef CONFIG_BLK_DEV_INITRD /*
arch/microblaze/mm/init.c renamed from arch/microblaze/mm/init.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +5 -4
@@ -109,13 +109,14 @@ static void __init paging_init(void)void __init setup_memory (void ) {#ifndef CONFIG_MMU -struct memblock_region *reg;u32 kernel_align_start, kernel_align_size; +phys_addr_t start, end; +u64 i;/* Find main memory where is the kernel */ -for_each_memblock (memory, reg) { - memory_start = (u32 )reg->base; - lowmem_size = reg->size; +for_each_mem_range (i, &start, &end) { + memory_start = start; + lowmem_size = end - start;if ((memory_start <= (u32 )_text) && ((u32 )_text <= (memory_start + lowmem_size -1 ))) { memory_size = lowmem_size;
arch/mips/cavium-octeon/dma-octeon.c renamed from arch/mips/cavium-octeon/dma-octeon.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +7 -7
@@ -190,25 +190,25 @@ char *octeon_swiotlb;void __init plat_swiotlb_setup (void ) { -struct memblock_region *mem; +phys_addr_t start, end;phys_addr_t max_addr;phys_addr_t addr_size;size_t swiotlbsize;unsigned long swiotlb_nslabs; +u64 i; max_addr =0 ; addr_size =0 ; -for_each_memblock (memory, mem) { +for_each_mem_range (i, &start, &end) {/* These addresses map low for PCI. */ -if (mem->base >0x410000000ull && !OCTEON_IS_OCTEON2 ()) +if (start >0x410000000ull && !OCTEON_IS_OCTEON2 ())continue ; - addr_size += mem->size; - -if (max_addr < mem->base + mem->size) - max_addr = mem->base + mem->size; + addr_size += (end - start); +if (max_addr < end) + max_addr = end; } swiotlbsize =PAGE_SIZE ;
arch/mips/kernel/setup.c renamed from arch/mips/kernel/setup.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +15 -16
@@ -300,8 +300,9 @@ static void __init bootmem_init(void)static void __init bootmem_init (void ) { -struct memblock_region *mem;phys_addr_t ramstart, ramend; +phys_addr_t start, end; +u64 i; ramstart =memblock_start_of_DRAM (); ramend =memblock_end_of_DRAM ();
@@ -338,18 +339,13 @@ static void __init bootmem_init(void) min_low_pfn =ARCH_PFN_OFFSET ; max_pfn =PFN_DOWN (ramend); -for_each_memblock (memory, mem) { -unsigned long start =memblock_region_memory_base_pfn (mem); -unsigned long end =memblock_region_memory_end_pfn (mem); - +for_each_mem_range (i, &start, &end) {/* * Skip highmem here so we get an accurate max_low_pfn if low * memory stops short of high memory. * If the region overlaps HIGHMEM_START, end is clipped so * max_pfn excludes the highmem portion. */ -if (memblock_is_nomap (mem)) -continue ;if (start >=PFN_DOWN (HIGHMEM_START ))continue ;if (end >PFN_DOWN (HIGHMEM_START ))
@@ -450,13 +446,12 @@ early_param("memmap", early_parse_memmapunsigned long setup_elfcorehdr, setup_elfcorehdr_size;static int __init early_parse_elfcorehdr (char *p) { -struct memblock_region *mem; +phys_addr_t start, end; +u64 i; setup_elfcorehdr =memparse (p, &p); -for_each_memblock (memory, mem) { -unsigned long start = mem->base; -unsigned long end = start + mem->size; +for_each_mem_range (i, &start, &end) {if (setup_elfcorehdr >= start && setup_elfcorehdr < end) {/* * Reserve from the elf core header to the end of
@@ -720,7 +715,8 @@ static void __init arch_mem_init(char **static void __init resource_init (void ) { -struct memblock_region *region; +phys_addr_t start, end; +u64 i;if (UNCAC_BASE !=IO_BASE )return ;
@@ -732,9 +728,7 @@ static void __init resource_init(void) bss_resource.start =__pa_symbol (&__bss_start); bss_resource.end =__pa_symbol (&__bss_stop) -1 ; -for_each_memblock (memory, region) { -phys_addr_t start =PFN_PHYS (memblock_region_memory_base_pfn (region)); -phys_addr_t end =PFN_PHYS (memblock_region_memory_end_pfn (region)) -1 ; +for_each_mem_range (i, &start, &end) {struct resource *res; res =memblock_alloc (sizeof (struct resource ),SMP_CACHE_BYTES );
@@ -743,7 +737,12 @@ static void __init resource_init(void) sizeof(struct resource)); res->start = start; - res->end = end; +/* +* In memblock, end points to the first byte after the +* range while in resourses, end points to the last byte in +* the range. +*/ + res->end = end -1 ; res->flags =IORESOURCE_SYSTEM_RAM |IORESOURCE_BUSY ; res->name ="System RAM" ;
arch/openrisc/mm/init.c renamed from arch/openrisc/mm/init.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +5 -3
@@ -64,6 +64,7 @@ extern const char _s_kernel_ro[], _e_ker*/ static void __init map_ram (void ) { +phys_addr_t start, end;unsigned long v, p, e;pgprot_t prot;pgd_t *pge;
@@ -71,6 +72,7 @@ static void __init map_ram(void)pud_t *pue;pmd_t *pme;pte_t *pte; +u64 i;/* These mark extents of read-only kernel pages... * ...from vmlinux.lds.S */
@@ -78,9 +80,9 @@ static void __init map_ram(void) v =PAGE_OFFSET ; -for_each_memblock (memory, region) { - p = (u32 ) region->base &PAGE_MASK ; - e = p + (u32 ) region->size; +for_each_mem_range (i, &start, &end) { + p = (u32 ) start &PAGE_MASK ; + e = (u32 ) end; v = (u32 )__va (p); pge =pgd_offset_k (v);
arch/powerpc/kernel/fadump.c renamed from arch/powerpc/kernel/fadump.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +23 -27
108 lines not on this page · raw message
arch/powerpc/kexec/file_load_64.c renamed from arch/powerpc/kexec/file_load_64.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +4 -6
@@ -138,15 +138,13 @@ out:*/ static int get_crash_memory_ranges (struct crash_mem **mem_ranges) { -struct memblock_region *reg; +phys_addr_t base, end;struct crash_mem *tmem; +u64 i;int ret; -for_each_memblock (memory, reg) { -u64 base, size; - - base = (u64 )reg->base; - size = (u64 )reg->size; +for_each_mem_range (i, &base, &end) { +u64 size = end - base;/* Skip backup memory region, which needs a separate entry */ if (base ==BACKUP_SRC_START ) {
arch/powerpc/mm/book3s64/hash_utils.c renamed from arch/powerpc/mm/book3s64/hash_utils.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +8 -8
@@ -7,7 +7,7 @@* * SMP scalability work: * Copyright (C) 2001 Anton Blanchard <[email protected]>, IBM -* +* * Module name: htab.c * * Description:
@@ -867,8 +867,8 @@ static void __init htab_initialize(void)unsigned long table;unsigned long pteg_count;unsigned long prot; -unsigned long base =0 , size =0 ; -struct memblock_region *reg; +phys_addr_t base =0 , size =0 , end; +u64 i;DBG (" -> htab_initialize()\n" );
@@ -884,7 +884,7 @@ static void __init htab_initialize(void)/* * Calculate the required size of the htab. We want the number of * PTEGs to equal one half the number of real pages. -*/ +*/ htab_size_bytes =htab_get_table_size (); pteg_count = htab_size_bytes >>7 ;
@@ -894,7 +894,7 @@ static void __init htab_initialize(void) firmware_has_feature(FW_FEATURE_PS3_LV1)) {/* Using a hypervisor which owns the htab */ htab_address =NULL ; - _SDR1 =0 ; + _SDR1 =0 ;#ifdef CONFIG_FA_DUMP /* * If firmware assisted dump is active firmware preserves
@@ -960,9 +960,9 @@ static void __init htab_initialize(void)#endif /* CONFIG_DEBUG_PAGEALLOC */ /* create bolted the linear mapping in the hash table */ -for_each_memblock (memory, reg) { - base = (unsigned long )__va (reg->base); - size = reg->size; +for_each_mem_range (i, &base, &end) { + size = end - base; + base = (unsigned long )__va (base);DBG ("creating mapping for region: %lx..%lx (prot: %lx)\n" , base, size, prot);
arch/powerpc/mm/book3s64/radix_pgtable.c renamed from arch/powerpc/mm/book3s64/radix_pgtable.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +5 -5
@@ -329,7 +329,8 @@ static int __meminit create_physical_mapstatic void __init radix_init_pgtable (void ) {unsigned long rts_field; -struct memblock_region *reg; +phys_addr_t start, end; +u64 i;/* We don't support slb for radix */ mmu_slb_size =0 ;
@@ -337,20 +338,19 @@ static void __init radix_init_pgtable(vo/* * Create the linear mapping */ -for_each_memblock (memory, reg) { +for_each_mem_range (i, &start, &end) {/* * The memblock allocator is up at this point, so the * page tables will be allocated within the range. No * need or a node (which we don't have yet). */ -if ((reg->base + reg->size) >=RADIX_VMALLOC_START ) { +if (end >=RADIX_VMALLOC_START ) {pr_warn ("Outside the supported range\n" );continue ; } -WARN_ON (create_physical_mapping (reg->base, - reg->base + reg->size, +WARN_ON (create_physical_mapping (start, end, radix_mem_block_size,-1 ,PAGE_KERNEL )); }
arch/powerpc/mm/kasan/kasan_init_32.c renamed from arch/powerpc/mm/kasan/kasan_init_32.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +4 -4
@@ -138,11 +138,11 @@ void __init kasan_mmu_init(void)void __init kasan_init (void ) { -struct memblock_region *reg; +phys_addr_t base, end; +u64 i; -for_each_memblock (memory, reg) { -phys_addr_t base = reg->base; -phys_addr_t top =min (base + reg->size, total_lowmem); +for_each_mem_range (i, &base, &end) { +phys_addr_t top =min (end, total_lowmem);int ret;if (base >= top)
arch/powerpc/mm/mem.c renamed from arch/powerpc/mm/mem.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +10 -6
@@ -585,20 +585,24 @@ void flush_icache_user_page(struct vm_ar*/ static int __init add_system_ram_resources (void ) { -struct memblock_region *reg; +phys_addr_t start, end; +u64 i; -for_each_memblock (memory, reg) { +for_each_mem_range (i, &start, &end) {struct resource *res; -unsigned long base = reg->base; -unsigned long size = reg->size; res =kzalloc (sizeof (struct resource ),GFP_KERNEL );WARN_ON (!res);if (res) { res->name ="System RAM" ; - res->start = base; - res->end = base + size -1 ; + res->start = start; +/* +* In memblock, end points to the first byte after +* the range while in resourses, end points to the +* last byte in the range. +*/ + res->end = end -1 ; res->flags =IORESOURCE_SYSTEM_RAM |IORESOURCE_BUSY ;WARN_ON (request_resource (&iomem_resource, res) <0 ); }
arch/powerpc/mm/pgtable_32.c renamed from arch/powerpc/mm/pgtable_32.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +4 -4
@@ -123,11 +123,11 @@ static void __init __mapin_ram_chunk(unsvoid __init mapin_ram (void ) { -struct memblock_region *reg; +phys_addr_t base, end; +u64 i; -for_each_memblock (memory, reg) { -phys_addr_t base = reg->base; -phys_addr_t top =min (base + reg->size, total_lowmem); +for_each_mem_range (i, &base, &end) { +phys_addr_t top =min (end, total_lowmem);if (base >= top)continue ;
arch/riscv/mm/init.c renamed from arch/riscv/mm/init.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +10 -15
@@ -145,21 +145,21 @@ static phys_addr_t dtb_early_pa __initdavoid __init setup_bootmem (void ) { -struct memblock_region *reg;phys_addr_t mem_size =0 ;phys_addr_t total_mem =0 ; -phys_addr_t mem_start, end =0 ; +phys_addr_t mem_start, start, end =0 ;phys_addr_t vmlinux_end =__pa_symbol (&_end);phys_addr_t vmlinux_start =__pa_symbol (&_start); +u64 i;/* Find the memory region containing the kernel */ -for_each_memblock (memory, reg) { - end = reg->base + reg->size; +for_each_mem_range (i, &start, &end) { +phys_addr_t size = end - start;if (!total_mem) - mem_start = reg->base; -if (reg->base <= vmlinux_start && vmlinux_end <= end) -BUG_ON (reg->size ==0 ); - total_mem = total_mem + reg->size; + mem_start = start; +if (start <= vmlinux_start && vmlinux_end <= end) +BUG_ON (size ==0 ); + total_mem = total_mem + size; }/*
@@ -455,7 +455,7 @@ static void __init setup_vm_final(void) {uintptr_t va, map_size;phys_addr_t pa, start, end; -struct memblock_region *reg; +u64 i;/* Set mmu_enabled flag */ mmu_enabled = true;
@@ -466,14 +466,9 @@ static void __init setup_vm_final(void) PGDIR_SIZE, PAGE_TABLE);/* Map all memory banks */ -for_each_memblock (memory, reg) { - start = reg->base; - end = start + reg->size; - +for_each_mem_range (i, &start, &end) {if (start >= end)break ; -if (memblock_is_nomap (reg)) -continue ;if (start <=__pa (PAGE_OFFSET ) &&__pa (PAGE_OFFSET ) < end) start =__pa (PAGE_OFFSET );
arch/riscv/mm/kasan_init.c renamed from arch/riscv/mm/kasan_init.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +5 -5
@@ -85,16 +85,16 @@ static void __init populate(void *start,void __init kasan_init (void ) { -struct memblock_region *reg; -unsigned long i; +phys_addr_t _start, _end; +u64 i;kasan_populate_early_shadow ((void *)KASAN_SHADOW_START , (void *)kasan_mem_to_shadow ((void *)VMALLOC_END )); -for_each_memblock (memory, reg) { -void *start = (void *)__va (reg->base); -void *end = (void *)__va (reg->base + reg->size); +for_each_mem_range (i, &_start, &_end) { +void *start = (void *)_start; +void *end = (void *)_end;if (start >= end)break ;
arch/s390/kernel/setup.c renamed from arch/s390/kernel/setup.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +15 -8
@@ -484,8 +484,9 @@ static struct resource __initdata *standstatic void __init setup_resources (void ) {struct resource *res, *std_res, *sub_res; -struct memblock_region *reg; +phys_addr_t start, end;int j; +u64 i; code_resource.start = (unsigned long ) _text; code_resource.end = (unsigned long ) _etext -1 ;
@@ -494,7 +495,7 @@ static void __init setup_resources(void) bss_resource.start = (unsigned long ) __bss_start; bss_resource.end = (unsigned long ) __bss_stop -1 ; -for_each_memblock (memory, reg) { +for_each_mem_range (i, &start, &end) { res =memblock_alloc (sizeof (*res),8 );if (!res) panic("%s: Failed to allocate %zu bytes align=0x%x\n",
@@ -502,8 +503,13 @@ static void __init setup_resources(void) res->flags =IORESOURCE_BUSY |IORESOURCE_SYSTEM_RAM ; res->name ="System RAM" ; - res->start = reg->base; - res->end = reg->base + reg->size -1 ; + res->start = start; +/* +* In memblock, end points to the first byte after the +* range while in resourses, end points to the last byte in +* the range. +*/ + res->end = end -1 ;request_resource (&iomem_resource, res);for (j =0 ; j <ARRAY_SIZE (standard_resources); j++) {
@@ -819,14 +825,15 @@ static void __init reserve_kernel(void)static void __init setup_memory (void ) { -struct memblock_region *reg; +phys_addr_t start, end; +u64 i;/* * Init storage key for present memory */ -for_each_memblock (memory, reg) { -storage_key_init_range (reg->base, reg->base + reg->size); - } +for_each_mem_range (i, &start, &end) +storage_key_init_range (start, end); +psw_set_key (PAGE_DEFAULT_KEY );/* Only cosmetics */
arch/s390/mm/vmem.c renamed from arch/s390/mm/vmem.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +4 -3
@@ -555,10 +555,11 @@ int vmem_add_mapping(unsigned long start*/ void __init vmem_map_init (void ) { -struct memblock_region *reg; +phys_addr_t base, end; +u64 i; -for_each_memblock (memory, reg) -vmem_add_range (reg->base, reg->size); +for_each_mem_range (i, &base, &end) +vmem_add_range (base, end - base);__set_memory ((unsigned long )_stext, (unsigned long )(_etext - _stext) >>PAGE_SHIFT ,SET_MEMORY_RO |SET_MEMORY_X );
arch/sparc/mm/init_64.c renamed from arch/sparc/mm/init_64.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +4 -8
@@ -1192,18 +1192,14 @@ int of_node_to_nid(struct device_node *dstatic void __init add_node_ranges (void ) { -struct memblock_region *reg; +phys_addr_t start, end;unsigned long prev_max; +u64 i; memblock_resized: prev_max = memblock.memory.max; -for_each_memblock (memory, reg) { -unsigned long size = reg->size; -unsigned long start, end; - - start = reg->base; - end = start + size; +for_each_mem_range (i, &start, &end) {while (start < end) {unsigned long this_end;int nid;
@@ -1211,7 +1207,7 @@ memblock_resized: this_end =memblock_nid_range (start, end, &nid);numadbg ("Setting memblock NUMA node nid[%d] " -"start[%lx] end[%lx]\n" , +"start[%llx] end[%lx]\n" , nid, start, this_end); memblock_set_node(start, this_end - start,
drivers/bus/mvebu-mbus.c renamed from drivers/bus/mvebu-mbus.c~arch-drivers-replace-for_each_membock-with-for_each_mem_range +6 -6
@@ -610,23 +610,23 @@ static unsigned int armada_xp_mbus_win_rstatic void __init mvebu_mbus_find_bridge_hole (uint64_t *start,uint64_t *end) { -struct memblock_region *r; -uint64_t s =0 ; +phys_addr_t reg_start, reg_end; +uint64_t i, s =0 ; -for_each_memblock (memory, r) { +for_each_mem_range (i, ®_start, ®_end) {/* * This part of the memory is above 4 GB, so we don't * care for the MBus bridge hole. */ -if (r->base >=0x100000000ULL ) +if (reg_start >=0x100000000ULL )continue ;/* * The MBus bridge hole is at the end of the RAM under * the 4 GB limit. */ -if (r->base + r->size > s) - s = r->base + r->size; +if (reg_end > s) + s = reg_end; } *start = s;
_
Andrew Morton
maintainerPATCH 174/181 x86/setup: simplify initrd relocation and reservation
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: x86/setup: simplify initrd relocation and reservation
Currently, initrd image is reserved very early during setup and then it might be relocated and re-reserved after the initial physical memory mapping is created. The "late" reservation of memblock verifies that mapped memory size exceeds the size of initrd, then checks whether the relocation required and, if yes, relocates inirtd to a new memory allocated from memblock and frees the old location.
The check for memory size is excessive as memblock allocation will anyway fail if there is not enough memory. Besides, there is no point to allocate memory from memblock using memblock_find_in_range() + memblock_reserve() when there exists memblock_phys_alloc_range() with required functionality.
Remove the redundant check and simplify memblock allocation.
---
arch/x86/kernel/setup.c | 16 +++------------- 1 file changed, 3 insertions(+), 13 deletions(-)
arch/x86/kernel/setup.c renamed from arch/x86/kernel/setup.c~x86-setup-simplify-initrd-relocation-and-reservation +3 -13
@@ -264,16 +264,12 @@ static void __init relocate_initrd(void)u64 area_size =PAGE_ALIGN (ramdisk_size);/* We need to move the initrd down into directly mapped mem */ - relocated_ramdisk =memblock_find_in_range (0 ,PFN_PHYS (max_pfn_mapped), - area_size,PAGE_SIZE ); - + relocated_ramdisk =memblock_phys_alloc_range (area_size,PAGE_SIZE ,0 , +PFN_PHYS (max_pfn_mapped));if (!relocated_ramdisk)panic ("Cannot find place for new RAMDISK of size %lld\n" , ramdisk_size); -/* Note: this includes all the mem currently occupied by -the initrd, we rely on that fact to keep the data intact. */ -memblock_reserve (relocated_ramdisk, area_size); initrd_start = relocated_ramdisk +PAGE_OFFSET ; initrd_end = initrd_start + ramdisk_size; printk(KERN_INFO "Allocated new RAMDISK: [mem %#010llx-%#010llx]\n",
@@ -300,13 +296,13 @@ static void __init early_reserve_initrd(memblock_reserve (ramdisk_image, ramdisk_end - ramdisk_image); } +static void __init reserve_initrd (void ) {/* Assume only end is not page aligned */ u64 ramdisk_image =get_ramdisk_image ();u64 ramdisk_size =get_ramdisk_size ();u64 ramdisk_end =PAGE_ALIGN (ramdisk_image + ramdisk_size); -u64 mapped_size;if (!boot_params.hdr.type_of_loader || !ramdisk_image || !ramdisk_size)
@@ -314,12 +310,6 @@ static void __init reserve_initrd(void) initrd_start =0 ; - mapped_size =memblock_mem_size (max_pfn_mapped); -if (ramdisk_size >= (mapped_size>>1 )) -panic ("initrd too large to handle, " -"disabling initrd (%lld needed, %lld available)\n" , - ramdisk_size, mapped_size>>1 ); -printk (KERN_INFO "RAMDISK: [mem %#010llx-%#010llx]\n" , ramdisk_image, ramdisk_end -1 );
_
Andrew Morton
maintainerPATCH 175/181 x86/setup: simplify reserve_crashkernel()
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: x86/setup: simplify reserve_crashkernel()
* Replace magic numbers with defines
* Replace memblock_find_in_range() + memblock_reserve() with memblock_phys_alloc_range()
* Stop checking for low memory size in reserve_crashkernel_low(). The allocation from limited range will anyway fail if there is no enough memory, so there is no need for extra traversal of memblock.memory
---
arch/x86/kernel/setup.c | 40 +++++++++++++------------------------- 1 file changed, 14 insertions(+), 26 deletions(-)
arch/x86/kernel/setup.c renamed from arch/x86/kernel/setup.c~x86-setup-simplify-reserve_crashkernel +14 -26
@@ -421,13 +421,13 @@ static int __init reserve_crashkernel_lo {#ifdef CONFIG_X86_64 unsigned long long base, low_base =0 , low_size =0 ; -unsigned long total_low_mem; +unsigned long low_mem_limit;int ret; - total_low_mem =memblock_mem_size (1UL << (32 -PAGE_SHIFT )); + low_mem_limit =min (memblock_phys_mem_size (),CRASH_ADDR_LOW_MAX );/* crashkernel=Y,low */ - ret =parse_crashkernel_low (boot_command_line, total_low_mem, &low_size, &base); + ret =parse_crashkernel_low (boot_command_line, low_mem_limit, &low_size, &base);if (ret) {/* * two parts from kernel/dma/swiotlb.c:
@@ -445,23 +445,17 @@ static int __init reserve_crashkernel_loreturn 0 ; } - low_base =memblock_find_in_range (0 ,1ULL <<32 , low_size,CRASH_ALIGN ); + low_base =memblock_phys_alloc_range (low_size,CRASH_ALIGN ,0 ,CRASH_ADDR_LOW_MAX );if (!low_base) {pr_err ("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n" , (unsigned long )(low_size >>20 ));return -ENOMEM ; } - ret =memblock_reserve (low_base, low_size); -if (ret) { -pr_err ("%s: Error reserving crashkernel low memblock.\n" , __func__); -return ret; - } - -pr_info ("Reserving %ldMB of low memory at %ldMB for crashkernel (System low RAM: %ldMB)\n" , +pr_info ("Reserving %ldMB of low memory at %ldMB for crashkernel (low RAM limit: %ldMB)\n" , (unsigned long )(low_size >>20 ), (unsigned long )(low_base >>20 ), - (unsigned long )(total_low_mem >>20 )); + (unsigned long )(low_mem_limit >>20 )); crashk_low_res.start = low_base; crashk_low_res.end = low_base + low_size -1 ;
@@ -505,13 +499,13 @@ static void __init reserve_crashkernel(v* unless "crashkernel=size[KMG],high" is specified. */ if (!high) - crash_base =memblock_find_in_range (CRASH_ALIGN , -CRASH_ADDR_LOW_MAX , - crash_size,CRASH_ALIGN ); + crash_base =memblock_phys_alloc_range (crash_size, +CRASH_ALIGN ,CRASH_ALIGN , +CRASH_ADDR_LOW_MAX );if (!crash_base) - crash_base =memblock_find_in_range (CRASH_ALIGN , -CRASH_ADDR_HIGH_MAX , - crash_size,CRASH_ALIGN ); + crash_base =memblock_phys_alloc_range (crash_size, +CRASH_ALIGN ,CRASH_ALIGN , +CRASH_ADDR_HIGH_MAX );if (!crash_base) {pr_info ("crashkernel reservation failed - No suitable area found.\n" );return ;
@@ -519,19 +513,13 @@ static void __init reserve_crashkernel(v }else {unsigned long long start; - start =memblock_find_in_range (crash_base, - crash_base + crash_size, - crash_size,1 <<20 ); + start =memblock_phys_alloc_range (crash_size,SZ_1M , crash_base, + crash_base + crash_size);if (start != crash_base) {pr_info ("crashkernel reservation failed - memory is in use.\n" );return ; } } - ret =memblock_reserve (crash_base, crash_size); -if (ret) { -pr_err ("%s: Error reserving crashkernel memblock.\n" , __func__); -return ; - }if (crash_base >= (1ULL <<32 ) &&reserve_crashkernel_low ()) {memblock_free (crash_base, crash_size);
_
Andrew Morton
maintainerPATCH 176/181 memblock: remove unused memblock_mem_size()
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: memblock: remove unused memblock_mem_size()
The only user of memblock_mem_size() was x86 setup code, it is gone now and memblock_mem_size() funciton can be removed.
---
include/linux/memblock.h | 1 - mm/memblock.c | 15 --------------- 2 files changed, 16 deletions(-)
include/linux/memblock.h renamed from include/linux/memblock.h~memblock-remove-unused-memblock_mem_size +0 -1
@@ -481,7 +481,6 @@ static inline bool memblock_bottom_up(vophys_addr_t memblock_phys_mem_size (void );phys_addr_t memblock_reserved_size (void ); -phys_addr_t memblock_mem_size (unsigned long limit_pfn);phys_addr_t memblock_start_of_DRAM (void );phys_addr_t memblock_end_of_DRAM (void );void memblock_enforce_memory_limit (phys_addr_t memory_limit);
mm/memblock.c renamed from mm/memblock.c~memblock-remove-unused-memblock_mem_size +0 -15
@@ -1660,21 +1660,6 @@ phys_addr_t __init_memblock memblock_resreturn memblock.reserved.total_size; } -phys_addr_t __init memblock_mem_size (unsigned long limit_pfn) -{ -unsigned long pages =0 ; -unsigned long start_pfn, end_pfn; -int i; - -for_each_mem_pfn_range (i,MAX_NUMNODES , &start_pfn, &end_pfn,NULL ) { - start_pfn = min_t(unsigned long, start_pfn, limit_pfn); - end_pfn = min_t(unsigned long, end_pfn, limit_pfn); - pages += end_pfn - start_pfn; - } - -return PFN_PHYS (pages); -} -/* lowest address */ phys_addr_t __init_memblock memblock_start_of_DRAM(void) {
_
Andrew Morton
maintainerPATCH 177/181 memblock: implement for_each_reserved_mem_region() using __next_mem_region()
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: memblock: implement for_each_reserved_mem_region() using __next_mem_region()
Iteration over memblock.reserved with for_each_reserved_mem_region() used __next_reserved_mem_region() that implemented a subset of __next_mem_region().
Use __for_each_mem_range() and, essentially, __next_mem_region() with appropriate parameters to reduce code duplication.
While on it, rename for_each_reserved_mem_region() to for_each_reserved_mem_range() for consistency.
---
.clang-format | 2 - arch/arm64/kernel/setup.c | 2 - drivers/irqchip/irq-gic-v3-its.c | 2 - include/linux/memblock.h | 12 ++---- mm/memblock.c | 56 ++++++++++------------------- 5 files changed, 27 insertions(+), 47 deletions(-)
arch/arm64/kernel/setup.c renamed from arch/arm64/kernel/setup.c~memblock-implement-for_each_reserved_mem_region-using-__next_mem_region +1 -1
@@ -257,7 +257,7 @@ static int __init reserve_memblock_reserif (!memblock_is_region_reserved (mem->start, mem_size))continue ; -for_each_reserved_mem_region (j, &r_start, &r_end) { +for_each_reserved_mem_range (j, &r_start, &r_end) {resource_size_t start, end; start =max (PFN_PHYS (PFN_DOWN (r_start)), mem->start);
.clang-format renamed from .clang-format~memblock-implement-for_each_reserved_mem_region-using-__next_mem_region +1 -1
@@ -273,7 +273,7 @@ ForEachMacros: - 'for_each_registered_fb' - 'for_each_requested_gpio' - 'for_each_requested_gpio_in_range' - - 'for_each_reserved_mem_region' + - 'for_each_reserved_mem_range' - 'for_each_rtd_codec_dais' - 'for_each_rtd_codec_dais_rollback' - 'for_each_rtd_components'
drivers/irqchip/irq-gic-v3-its.c renamed from drivers/irqchip/irq-gic-v3-its.c~memblock-implement-for_each_reserved_mem_region-using-__next_mem_region +1 -1
@@ -2198,7 +2198,7 @@ static bool gic_check_reserved_range(phy addr_end = addr + size -1 ; -for_each_reserved_mem_region (i, &start, &end) { +for_each_reserved_mem_range (i, &start, &end) {if (addr >= start && addr_end <= end)return true; }
include/linux/memblock.h renamed from include/linux/memblock.h~memblock-implement-for_each_reserved_mem_region-using-__next_mem_region +4 -8
@@ -132,9 +132,6 @@ void __next_mem_range_rev(u64 *idx, int struct memblock_type *type_b, phys_addr_t *out_start, phys_addr_t *out_end, int *out_nid); -void __next_reserved_mem_region (u64 *idx,phys_addr_t *out_start, -phys_addr_t *out_end); -void __memblock_free_late (phys_addr_t base,phys_addr_t size);#ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP
@@ -224,7 +221,7 @@ static inline void __next_physmem_range( MEMBLOCK_NONE, p_start, p_end, NULL)/** -* for_each_reserved_mem_region - iterate over all reserved memblock areas +* for_each_reserved_mem_range - iterate over all reserved memblock areas * @i: u64 used as loop variable * @p_start: ptr to phys_addr_t for start address of the range, can be %NULL * @p_end: ptr to phys_addr_t for end address of the range, can be %NULL
@@ -232,10 +229,9 @@ static inline void __next_physmem_range(* Walks over reserved areas of memblock. Available as soon as memblock * is initialized. */ -#define for_each_reserved_mem_region (i, p_start, p_end) \ - for (i = 0UL, __next_reserved_mem_region(&i, p_start, p_end); \ - i != (u64)ULLONG_MAX; \ - __next_reserved_mem_region(&i, p_start, p_end)) +#define for_each_reserved_mem_range (i, p_start, p_end) \ + __for_each_mem_range(i, &memblock.reserved, NULL, NUMA_NO_NODE, \ + MEMBLOCK_NONE, p_start, p_end, NULL)static inline bool memblock_is_hotpluggable (struct memblock_region *m) {
mm/memblock.c renamed from mm/memblock.c~memblock-implement-for_each_reserved_mem_region-using-__next_mem_region +20 -36
95 lines not on this page · raw message
_
Andrew Morton
maintainerPATCH 178/181 memblock: use separate iterators for memory and reserved regions
In reply to Andrew MortonFrom: Mike Rapoport <[email protected]>
Subject: memblock: use separate iterators for memory and reserved regions
for_each_memblock() is used to iterate over memblock.memory in a few places that use data from memblock_region rather than the memory ranges.
Introduce separate for_each_mem_region() and for_each_reserved_mem_region() to improve encapsulation of memblock internals from its users.
---
.clang-format | 3 ++- arch/arm64/kernel/setup.c | 2 +- arch/arm64/mm/numa.c | 2 +- arch/mips/netlogic/xlp/setup.c | 2 +- arch/riscv/mm/init.c | 2 +- arch/x86/mm/numa.c | 2 +- include/linux/memblock.h | 19 ++++++++++++++++--- mm/memblock.c | 4 ++-- mm/page_alloc.c | 8 ++++---- 9 files changed, 29 insertions(+), 15 deletions(-)
arch/arm64/kernel/setup.c renamed from arch/arm64/kernel/setup.c~memblock-use-separate-iterators-for-memory-and-reserved-regions +1 -1
@@ -217,7 +217,7 @@ static void __init request_standard_resoif (!standard_resources)panic ("%s: Failed to allocate %zu bytes\n" , __func__, res_size); -for_each_memblock (memory, region) { +for_each_mem_region (region) { res = &standard_resources[i++];if (memblock_is_nomap (region)) { res->name ="reserved" ;
arch/arm64/mm/numa.c renamed from arch/arm64/mm/numa.c~memblock-use-separate-iterators-for-memory-and-reserved-regions +1 -1
@@ -354,7 +354,7 @@ static int __init numa_register_nodes(vostruct memblock_region *mblk;/* Check that valid nid is set to memblks */ -for_each_memblock (memory, mblk) { +for_each_mem_region (mblk) {int mblk_nid =memblock_get_region_node (mblk);if (mblk_nid ==NUMA_NO_NODE || mblk_nid >=MAX_NUMNODES ) {
arch/mips/netlogic/xlp/setup.c renamed from arch/mips/netlogic/xlp/setup.c~memblock-use-separate-iterators-for-memory-and-reserved-regions +1 -1
@@ -70,7 +70,7 @@ static void nlm_fixup_mem(void)const int pref_backup =512 ;struct memblock_region *mem; -for_each_memblock (memory, mem) { +for_each_mem_region (mem) {memblock_remove (mem->base + mem->size - pref_backup, pref_backup); }
arch/riscv/mm/init.c renamed from arch/riscv/mm/init.c~memblock-use-separate-iterators-for-memory-and-reserved-regions +1 -1
@@ -531,7 +531,7 @@ static void __init resource_init(void) {struct memblock_region *region; -for_each_memblock (memory, region) { +for_each_mem_region (region) {struct resource *res; res =memblock_alloc (sizeof (struct resource ),SMP_CACHE_BYTES );
arch/x86/mm/numa.c renamed from arch/x86/mm/numa.c~memblock-use-separate-iterators-for-memory-and-reserved-regions +1 -1
@@ -514,7 +514,7 @@ static void __init numa_clear_kernel_nod* memory ranges, because quirks such as trim_snb_memory() * reserve specific pages for Sandy Bridge graphics. ] */ -for_each_memblock (reserved, mb_region) { +for_each_reserved_mem_region (mb_region) {int nid =memblock_get_region_node (mb_region);if (nid !=MAX_NUMNODES )
.clang-format renamed from .clang-format~memblock-use-separate-iterators-for-memory-and-reserved-regions +2 -1
@@ -203,7 +203,7 @@ ForEachMacros: - 'for_each_matching_node' - 'for_each_matching_node_and_match' - 'for_each_member' - - 'for_each_memblock' + - 'for_each_mem_region' - 'for_each_memblock_type' - 'for_each_memcg_cache_index' - 'for_each_mem_pfn_range'
@@ -274,6 +274,7 @@ ForEachMacros: - 'for_each_requested_gpio' - 'for_each_requested_gpio_in_range' - 'for_each_reserved_mem_range' + - 'for_each_reserved_mem_region' - 'for_each_rtd_codec_dais' - 'for_each_rtd_codec_dais_rollback' - 'for_each_rtd_components'
include/linux/memblock.h renamed from include/linux/memblock.h~memblock-use-separate-iterators-for-memory-and-reserved-regions +16 -3
@@ -553,9 +553,22 @@ static inline unsigned long memblock_regreturn PFN_UP (reg->base + reg->size); } -#define for_each_memblock (memblock_type, region) \ - for (region = memblock.memblock_type.regions; \ - region < (memblock.memblock_type.regions + memblock.memblock_type.cnt); \ +/** +* for_each_mem_region - itereate over memory regions +* @region: loop variable +*/ +#define for_each_mem_region (region) \ + for (region = memblock.memory.regions; \ + region < (memblock.memory.regions + memblock.memory.cnt); \ + region++) + +/** +* for_each_reserved_mem_region - itereate over reserved memory regions +* @region: loop variable +*/ +#define for_each_reserved_mem_region (region) \ + for (region = memblock.reserved.regions; \ + region < (memblock.reserved.regions + memblock.reserved.cnt); \ region++) extern void *alloc_large_system_hash(const char *tablename,
mm/memblock.c renamed from mm/memblock.c~memblock-use-separate-iterators-for-memory-and-reserved-regions +2 -2
@@ -1667,7 +1667,7 @@ static phys_addr_t __init_memblock __fin* the memory memblock regions, if the @limit exceeds the total size * of those regions, max_addr will keep original value PHYS_ADDR_MAX */ -for_each_memblock (memory, r) { +for_each_mem_region (r) {if (limit <= r->size) { max_addr = r->base + limit;break ;
@@ -1837,7 +1837,7 @@ void __init_memblock memblock_trim_memorphys_addr_t start, end, orig_start, orig_end;struct memblock_region *r; -for_each_memblock (memory, r) { +for_each_mem_region (r) { orig_start = r->base; orig_end = r->base + r->size; start =round_up (orig_start, align);
mm/page_alloc.c renamed from mm/page_alloc.c~memblock-use-separate-iterators-for-memory-and-reserved-regions +4 -4
@@ -5961,7 +5961,7 @@ overlap_memmap_init(unsigned long zone,if (mirrored_kernelcore && zone ==ZONE_MOVABLE ) {if (!r || *pfn >=memblock_region_memory_end_pfn (r)) { -for_each_memblock (memory, r) { +for_each_mem_region (r) {if (*pfn <memblock_region_memory_end_pfn (r))break ; }
@@ -6546,7 +6546,7 @@ static unsigned long __init zone_absent_unsigned long start_pfn, end_pfn;struct memblock_region *r; -for_each_memblock (memory, r) { +for_each_mem_region (r) { start_pfn =clamp (memblock_region_memory_base_pfn (r), zone_start_pfn, zone_end_pfn); end_pfn = clamp(memblock_region_memory_end_pfn(r),
@@ -7140,7 +7140,7 @@ static void __init find_zone_movable_pfn* options. */ if (movable_node_is_enabled ()) { -for_each_memblock (memory, r) { +for_each_mem_region (r) {if (!memblock_is_hotpluggable (r))continue ;
@@ -7161,7 +7161,7 @@ static void __init find_zone_movable_pfnif (mirrored_kernelcore) {bool mem_below_4gb_not_mirrored = false; -for_each_memblock (memory, r) { +for_each_mem_region (r) {if (memblock_is_mirror (r))continue ;
_
Andrew Morton
maintainerPATCH 179/181 mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary
In reply to Andrew MortonFrom: Suren Baghdasaryan <[email protected]>
Subject: mm, oom_adj: don't loop through tasks in __set_oom_adj when not necessary
Currently __set_oom_adj loops through all processes in the system to keep oom_score_adj and oom_score_adj_min in sync between processes sharing their mm. This is done for any task with more that one mm_users, which includes processes with multiple threads (sharing mm and signals). However for such processes the loop is unnecessary because their signal structure is shared as well.
Android updates oom_score_adj whenever a tasks changes its role (background/foreground/...) or binds to/unbinds from a service, making it more/less important. Such operation can happen frequently. We noticed that updates to oom_score_adj became more expensive and after further investigation found out that the patch mentioned in "Fixes" introduced a regression. Using Pixel 4 with a typical Android workload, write time to oom_score_adj increased from ~3.57us to ~362us. Moreover this regression linearly depends on the number of multi-threaded processes running on the system.
Mark the mm with a new MMF_MULTIPROCESS flag bit when task is created with (CLONE_VM && !CLONE_THREAD && !CLONE_VFORK). Change __set_oom_adj to use MMF_MULTIPROCESS instead of mm_users to decide whether oom_score_adj update should be synchronized between multiple processes. To prevent races between clone() and __set_oom_adj(), when oom_score_adj of the process being cloned might be modified from userspace, we use oom_adj_mutex. Its scope is changed to global.
The combination of (CLONE_VM && !CLONE_THREAD) is rarely used except for the case of vfork(). To prevent performance regressions of vfork(), we skip taking oom_adj_mutex and setting MMF_MULTIPROCESS when CLONE_VFORK is specified. Clearing the MMF_MULTIPROCESS flag (when the last process sharing the mm exits) is left out of this patch to keep it simple and because it is believed that this threading model is rare. Should there ever be a need for optimizing that case as well, it can be done by hooking into the exit path, likely following the mm_update_next_owner pattern.
With the combination of (CLONE_VM && !CLONE_THREAD && !CLONE_VFORK) being quite rare, the regression is gone after the change is applied.
---
fs/proc/base.c | 3 +-- include/linux/oom.h | 1 + include/linux/sched/coredump.h | 1 + kernel/fork.c | 21 +++++++++++++++++++++ mm/oom_kill.c | 2 ++ 5 files changed, 26 insertions(+), 2 deletions(-)
fs/proc/base.c renamed from fs/proc/base.c~mm-oom_adj-dont-loop-through-tasks-in-__set_oom_adj-when-not-necessary +1 -2
@@ -1055,7 +1055,6 @@ static ssize_t oom_adj_read(struct filestatic int __set_oom_adj (struct file *file,int oom_adj,bool legacy) { -static DEFINE_MUTEX (oom_adj_mutex);struct mm_struct *mm =NULL ;struct task_struct *task;int err =0 ;
@@ -1095,7 +1094,7 @@ static int __set_oom_adj(struct file *fistruct task_struct *p =find_lock_task_mm (task);if (p) { -if (atomic_read (&p->mm->mm_users) >1 ) { +if (test_bit (MMF_MULTIPROCESS , &p->mm->flags)) { mm = p->mm;mmgrab (mm); }
include/linux/oom.h renamed from include/linux/oom.h~mm-oom_adj-dont-loop-through-tasks-in-__set_oom_adj-when-not-necessary +1 -0
@@ -55,6 +55,7 @@ struct oom_control { };extern struct mutex oom_lock; +extern struct mutex oom_adj_mutex;static inline void set_current_oom_origin (void ) {
include/linux/sched/coredump.h renamed from include/linux/sched/coredump.h~mm-oom_adj-dont-loop-through-tasks-in-__set_oom_adj-when-not-necessary +1 -0
@@ -72,6 +72,7 @@ static inline int get_dumpable(struct mm#define MMF_DISABLE_THP 24/* disable THP for all VMAs */ #define MMF_OOM_VICTIM 25/* mm is the oom victim */ #define MMF_OOM_REAP_QUEUED 26/* mm was queued for oom_reaper */ +#define MMF_MULTIPROCESS 27/* mm is shared between processes */ #define MMF_DISABLE_THP_MASK (1 << MMF_DISABLE_THP)#define MMF_INIT_MASK (MMF_DUMPABLE_MASK | MMF_DUMP_FILTER_MASK |\
kernel/fork.c renamed from kernel/fork.c~mm-oom_adj-dont-loop-through-tasks-in-__set_oom_adj-when-not-necessary +21 -0
@@ -1812,6 +1812,25 @@ static __always_inline void delayed_freefree_task (tsk); } +static void copy_oom_score_adj (u64 clone_flags,struct task_struct *tsk) +{ +/* Skip if kernel thread */ +if (!tsk->mm) +return ; + +/* Skip if spawning a thread or using vfork */ +if ((clone_flags & (CLONE_VM |CLONE_THREAD |CLONE_VFORK )) !=CLONE_VM ) +return ; + +/* We need to synchronize with __set_oom_adj */ +mutex_lock (&oom_adj_mutex); +set_bit (MMF_MULTIPROCESS , &tsk->mm->flags); +/* Update the values in case they were changed after copy_signal */ + tsk->signal->oom_score_adj = current->signal->oom_score_adj; + tsk->signal->oom_score_adj_min = current->signal->oom_score_adj_min; +mutex_unlock (&oom_adj_mutex); +} +/* * This creates a new process as a copy of the old one, * but does not actually start it yet.
@@ -2288,6 +2307,8 @@ static __latent_entropy struct task_strutrace_task_newtask (p, clone_flags);uprobe_copy_process (p, clone_flags); +copy_oom_score_adj (clone_flags, p); +return p; bad_fork_cancel_cgroup:
mm/oom_kill.c renamed from mm/oom_kill.c~mm-oom_adj-dont-loop-through-tasks-in-__set_oom_adj-when-not-necessary +2 -0
@@ -64,6 +64,8 @@ int sysctl_oom_dump_tasks = 1;* and mark_oom_victim */ DEFINE_MUTEX (oom_lock); +/* Serializes oom_score_adj and oom_score_adj_min updates */ +DEFINE_MUTEX (oom_adj_mutex);static inline bool is_memcg_oom (struct oom_control *oc) {
_
Andrew Morton
maintainerPATCH 180/181 mm/migrate: remove cpages-- in migrate_vma_finalize()
In reply to Andrew MortonFrom: Ralph Campbell <[email protected]>
Subject: mm/migrate: remove cpages-- in migrate_vma_finalize()
The variable struct migrate_vma->cpages is only used in migrate_vma_setup(). There is no need to decrement it in migrate_vma_finalize() since it is never checked.
---
mm/migrate.c | 1 - 1 file changed, 1 deletion(-)
mm/migrate.c renamed from mm/migrate.c~mm-migrate-remove-cpages-in-migrate_vma_finalize +0 -1
@@ -3077,7 +3077,6 @@ void migrate_vma_finalize(struct migrateremove_migration_ptes (page, newpage, false);unlock_page (page); - migrate->cpages--;if (is_zone_device_page (page))put_page (page);
_
Andrew Morton
maintainerPATCH 181/181 mm/migrate: remove obsolete comment about device public
In reply to Andrew MortonFrom: Ralph Campbell <[email protected]>
Subject: mm/migrate: remove obsolete comment about device public
Device public memory never had an in tree consumer and was removed in commit 25b2995a35b6 ("mm: remove MEMORY_DEVICE_PUBLIC support"). Delete the obsolete comment.
---
mm/migrate.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/migrate.c renamed from mm/migrate.c~mm-migrate-remove-obsolete-comment-about-device-public +1 -1
@@ -381,7 +381,7 @@ static int expected_page_refs(struct addint expected_count =1 ;/* -* Device public or private pages have an extra refcount as they are +* Device private pages have an extra refcount as they are * ZONE_DEVICE pages. */ expected_count +=is_device_private_page (page);
_
+ maintainers-jarkkosakkinen-linuxintelcom-jarkko-kernelorg.patch added to -mm tree
In reply to Andrew MortonThe patch titled
The patch titled
Subject: MAINTAINERS: [email protected] -> [email protected]
has been added to the -mm tree. Its filename is
maintainers-jarkkosakkinen-linuxintelcom-jarkko-kernelorg.patch
This patch should soon appear at
https://ozlabs.org/~akpm/mmots/broken-out/maintainers-jarkkosakkinen-linuxintelcom-jarkko-kernelorg.patch
and later at
https://ozlabs.org/~akpm/mmotm/broken-out/maintainers-jarkkosakkinen-linuxintelcom-jarkko-kernelorg.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated there every 3-4 working days
------------------------------------------------------
From: Jarkko Sakkinen <[email protected]>
Subject: MAINTAINERS: [email protected] -> [email protected]
Use @kernel.org address as the main communications end point. Update the corresponding M-entries and .mailmap (for git shortlog translation).
---
.mailmap | 1 + MAINTAINERS | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-)
.mailmap renamed from .mailmap~maintainers-jarkkosakkinen-linuxintelcom-jarkko-kernelorg +1 -0
@@ -133,6 +133,7 @@ James Ketrenos <jketreno@io.(none)> Jan Glauber <[email protected]> <[email protected]> Jan Glauber <[email protected]> <[email protected]> Jan Glauber <[email protected]> <[email protected]> +Jarkko Sakkinen <[email protected]> <[email protected]> Jason Gunthorpe <[email protected]> <[email protected]> Jason Gunthorpe <[email protected]> <[email protected]> Jason Gunthorpe <[email protected]> <[email protected]>
MAINTAINERS renamed from MAINTAINERS~maintainers-jarkkosakkinen-linuxintelcom-jarkko-kernelorg +3 -3
@@ -9701,7 +9701,7 @@ F: security/keys/encrypted-keys/ KEYS-TRUSTED M: James Bottomley <[email protected]> -M: Jarkko Sakkinen <[email protected]> +M: Jarkko Sakkinen <[email protected]> M: Mimi Zohar <[email protected]> L: [email protected] L: [email protected]
@@ -9713,7 +9713,7 @@ F: security/keys/trusted-keys/ KEYS/KEYRINGS M: David Howells <[email protected]> -M: Jarkko Sakkinen <[email protected]> +M: Jarkko Sakkinen <[email protected]> L: [email protected] S: Maintained F: Documentation/security/keys/core.rst
@@ -17684,7 +17684,7 @@ F: drivers/platform/x86/toshiba-wmi.c TPM DEVICE DRIVER M: Peter Huewe <[email protected]> -M: Jarkko Sakkinen <[email protected]> +M: Jarkko Sakkinen <[email protected]> R: Jason Gunthorpe <[email protected]> L: [email protected] S: Maintained
_
Patches currently in -mm which might be from [email protected] are
maintainers-jarkkosakkinen-linuxintelcom-jarkko-kernelorg.patch
+ mm-debug_vm_pgtable-avoid-none-pte-in-pte_clear_test-fix.patch added to -mm tree
In reply to Andrew MortonThe patch titled Subject: mm/debug_vm_pgtable: avoid kernel crash with riscv has been added to the -mm tree. Its filename is mm-debug_vm_pgtable-avoid-none-pte-in-pte_clear_test-fix.patch
The patch titled
Subject: mm/debug_vm_pgtable: avoid kernel crash with riscv
has been added to the -mm tree. Its filename is
mm-debug_vm_pgtable-avoid-none-pte-in-pte_clear_test-fix.patch
This patch should soon appear at
https://ozlabs.org/~akpm/mmots/broken-out/mm-debug_vm_pgtable-avoid-none-pte-in-pte_clear_test-fix.patch
and later at
https://ozlabs.org/~akpm/mmotm/broken-out/mm-debug_vm_pgtable-avoid-none-pte-in-pte_clear_test-fix.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated there every 3-4 working days
------------------------------------------------------
From: "Aneesh Kumar K.V" <[email protected]>
Subject: mm/debug_vm_pgtable: avoid kernel crash with riscv
It was observed that the usage of RANDOM_ORVALUE with a valid pte cause kernel crash on riscv. Comment out the usage of random value on riscv
---
mm/debug_vm_pgtable.c | 2 ++ 1 file changed, 2 insertions(+)
mm/debug_vm_pgtable.c renamed from mm/debug_vm_pgtable.c~mm-debug_vm_pgtable-avoid-none-pte-in-pte_clear_test-fix +2 -0
@@ -548,7 +548,9 @@ static void __init pte_clear_tests(strucpte_t pte =pfn_pte (pfn, prot);pr_debug ("Validating PTE clear\n" ); +#ifndef CONFIG_RISCV pte =__pte (pte_val (pte) |RANDOM_ORVALUE ); +#endif set_pte_at (mm, vaddr, ptep, pte);barrier ();pte_clear (mm, vaddr, ptep);
_
Patches currently in -mm which might be from [email protected] are
powerpc-mm-add-debug_vm-warn-for-pmd_clear.patch powerpc-mm-move-setting-pte-specific-flags-to-pfn_pte.patch mm-debug_vm_pgtable-ppc64-avoid-setting-top-bits-in-radom-value.patch mm-debug_vm_pgtables-hugevmap-use-the-arch-helper-to-identify-huge-vmap-support.patch mm-debug_vm_pgtable-savedwrite-enable-savedwrite-test-with-config_numa_balancing.patch mm-debug_vm_pgtable-thp-mark-the-pte-entry-huge-before-using-set_pmd-pud_at.patch mm-debug_vm_pgtable-set_pte-pmd-pud-dont-use-set__at-to-update-an-existing-pte-entry.patch mm-debug_vm_pgtable-locks-move-non-page-table-modifying-test-together.patch mm-debug_vm_pgtable-locks-take-correct-page-table-lock.patch mm-debug_vm_pgtable-thp-use-page-table-depost-withdraw-with-thp.patch mm-debug_vm_pgtable-pmd_clear-dont-use-pmd-pud_clear-on-pte-entries.patch mm-debug_vm_pgtable-hugetlb-disable-hugetlb-test-on-ppc64.patch mm-debug_vm_pgtable-avoid-none-pte-in-pte_clear_test.patch mm-debug_vm_pgtable-avoid-none-pte-in-pte_clear_test-fix.patch mm-debug_vm_pgtable-avoid-doing-memory-allocation-with-pgtable_t-mapped.patch
+ device-dax-kmem-fix-resource-release.patch added to -mm tree
In reply to Andrew MortonThe patch titled Subject: device-dax/kmem: fix resource release has been added to the -mm tree. Its filename is device-dax-kmem-fix-resource-release.patch
The patch titled
Subject: device-dax/kmem: fix resource release
has been added to the -mm tree. Its filename is
device-dax-kmem-fix-resource-release.patch
This patch should soon appear at
https://ozlabs.org/~akpm/mmots/broken-out/device-dax-kmem-fix-resource-release.patch
and later at
https://ozlabs.org/~akpm/mmotm/broken-out/device-dax-kmem-fix-resource-release.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated there every 3-4 working days
------------------------------------------------------
From: Dan Williams <[email protected]>
Subject: device-dax/kmem: fix resource release
The conversion to request_mem_region() is broken because it assumes that the range is marked busy prior to release. However, due to the way that the kmem driver manipulates the IORESOURCE_BUSY flag (clears it to let {add,remove}_memory() handle busy) it requires a manual release_resource() to perform cleanup.
Given that the actual 'struct resource *' needs to be recalled, not just the range, add that tracking to the kmem driver-data.
---
drivers/dax/kmem.c | 48 ++++++++++++++++++++++++++++++------------- 1 file changed, 34 insertions(+), 14 deletions(-)
drivers/dax/kmem.c renamed from drivers/dax/kmem.c~device-dax-kmem-fix-resource-release +34 -14
@@ -35,11 +35,17 @@ static int dax_kmem_range(struct dev_daxreturn 0 ; } +struct dax_kmem_data { +const char *res_name; +struct resource *res[]; +}; +static int dev_dax_kmem_probe (struct dev_dax *dev_dax) {struct device *dev = &dev_dax->dev; +struct dax_kmem_data *data; +int rc = -ENOMEM ;int i, mapped =0 ; -char *res_name;int numa_node;/*
@@ -55,14 +61,17 @@ static int dev_dax_kmem_probe(struct devreturn -EINVAL ; } - res_name =kstrdup (dev_name (dev),GFP_KERNEL ); -if (!res_name) + data =kzalloc (sizeof (*data) +sizeof (struct resource *) * dev_dax->nr_range,GFP_KERNEL ); +if (!data)return -ENOMEM ; + data->res_name =kstrdup (dev_name (dev),GFP_KERNEL ); +if (!data->res_name) + goto err_res_name; +for (i =0 ; i < dev_dax->nr_range; i++) {struct resource *res;struct range range; -int rc; rc =dax_kmem_range (dev_dax, i, &range);if (rc) {
@@ -72,7 +81,7 @@ static int dev_dax_kmem_probe(struct dev }/* Region is permanently reserved if hotremove fails. */ - res =request_mem_region (range.start,range_len (&range), res_name); + res =request_mem_region (range.start,range_len (&range), data->res_name);if (!res) {dev_warn (dev,"mapping%d: %#llx-%#llx could not reserve region\n" , i, range.start, range.end);
@@ -82,9 +91,10 @@ static int dev_dax_kmem_probe(struct dev*/ if (mapped)continue ; -kfree (res_name); -return -EBUSY ; + rc = -EBUSY ; + goto err_request_mem; } + data->res[i] = res;/* * Set flags appropriate for System RAM. Leave ..._BUSY clear
@@ -104,18 +114,25 @@ static int dev_dax_kmem_probe(struct devif (rc) {dev_warn (dev,"mapping%d: %#llx-%#llx memory add failed\n" , i, range.start, range.end); -release_mem_region (range.start,range_len (&range)); +release_resource (res); +kfree (res); + data->res[i] =NULL ;if (mapped)continue ; -kfree (res_name); -return rc; + goto err_request_mem; } mapped++; } -dev_set_drvdata (dev, res_name); +dev_set_drvdata (dev, data);return 0 ; + +err_request_mem: +kfree (data->res_name); +err_res_name: +kfree (data); +return rc; }#ifdef CONFIG_MEMORY_HOTREMOVE
@@ -123,7 +140,7 @@ static int dev_dax_kmem_remove(struct de {int i, success =0 ;struct device *dev = &dev_dax->dev; -const char *res_name =dev_get_drvdata (dev); +struct dax_kmem_data *data =dev_get_drvdata (dev);/* * We have one shot for removing memory, if some memory blocks were not
@@ -142,7 +159,9 @@ static int dev_dax_kmem_remove(struct de rc =remove_memory (dev_dax->target_node, range.start,range_len (&range));if (rc ==0 ) { -release_mem_region (range.start,range_len (&range)); +release_resource (data->res[i]); +kfree (data->res[i]); + data->res[i] =NULL ; success++;continue ; }
@@ -153,7 +172,8 @@ static int dev_dax_kmem_remove(struct de }if (success >= dev_dax->nr_range) { -kfree (res_name); +kfree (data->res_name); +kfree (data);dev_set_drvdata (dev,NULL ); }
_
Patches currently in -mm which might be from [email protected] are
device-dax-kmem-fix-resource-release.patch xen-unpopulated-alloc-consolidate-pgmap-manipulation.patch
+ xen-unpopulated-alloc-consolidate-pgmap-manipulation.patch added to -mm tree
In reply to Andrew MortonThe patch titled Subject: xen/unpopulated-alloc: consolidate pgmap manipulation has been added to the -mm tree. Its filename is xen-unpopulated-alloc-consolidate-pgmap-manipulation.patch
The patch titled
Subject: xen/unpopulated-alloc: consolidate pgmap manipulation
has been added to the -mm tree. Its filename is
xen-unpopulated-alloc-consolidate-pgmap-manipulation.patch
This patch should soon appear at
https://ozlabs.org/~akpm/mmots/broken-out/xen-unpopulated-alloc-consolidate-pgmap-manipulation.patch
and later at
https://ozlabs.org/~akpm/mmotm/broken-out/xen-unpopulated-alloc-consolidate-pgmap-manipulation.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated there every 3-4 working days
------------------------------------------------------
From: Dan Williams <[email protected]>
Subject: xen/unpopulated-alloc: consolidate pgmap manipulation
Cleanup fill_list() to keep all the pgmap manipulations in a single location of the function. Update the exit unwind path accordingly.
---
drivers/xen/unpopulated-alloc.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
drivers/xen/unpopulated-alloc.c renamed from drivers/xen/unpopulated-alloc.c~xen-unpopulated-alloc-consolidate-pgmap-manipulation +7 -7
@@ -27,11 +27,6 @@ static int fill_list(unsigned int nr_pagif (!res)return -ENOMEM ; - pgmap =kzalloc (sizeof (*pgmap),GFP_KERNEL ); -if (!pgmap) - goto err_pgmap; - - pgmap->type =MEMORY_DEVICE_GENERIC ; res->name ="Xen scratch" ; res->flags =IORESOURCE_MEM |IORESOURCE_BUSY ;
@@ -43,6 +38,11 @@ static int fill_list(unsigned int nr_pag goto err_resource; } + pgmap =kzalloc (sizeof (*pgmap),GFP_KERNEL ); +if (!pgmap) + goto err_pgmap; + + pgmap->type =MEMORY_DEVICE_GENERIC ; pgmap->range = (struct range ) { .start = res->start, .end = res->end,
@@ -91,10 +91,10 @@ static int fill_list(unsigned int nr_pagreturn 0 ; err_memremap: -release_resource (res); -err_resource:kfree (pgmap); err_pgmap: +release_resource (res); +err_resource:kfree (res);return ret; }
_
Patches currently in -mm which might be from [email protected] are
device-dax-kmem-fix-resource-release.patch xen-unpopulated-alloc-consolidate-pgmap-manipulation.patch