mm: add bitmap VMA flag helpers and convert all mmap_prepare to use them
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest13 patches 42 repliesIn review 11 of 13 applied
- v1 posting 2/212 patches 16 repliesSuperseded
- v1 posting 1/212 patches 3 repliesSuperseded
Changes in v2 from the cover letter
* Rebased on mm-unstable again, and then on my other series :)
* Removed sparse __private decoration - it is not doing us any good, and anybody accessing vma->flags.__vma_flags will be very obviously doing something wrong (TM) so we don't really need it. This will shut up the various bots reporting sparse issues :)
* Made functions which test VMA flags reference a const vma_flags_t * pointer for the flags being checked as per Jason.
* Updated entire series to affect this change, including userland tests.
* Fixed missed fixup in memfd_luo_retrieve() and folded into series. Turns out this doesn't get built with allmodconfig due to my arch not support.
* Fixed up issue with dev dax reported by Zi (thanks, much appreciated!)
Patches in v2 11 of 13 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/13 | mm/vma: remove __private sparse decoration from vma_flags_tinclude/ include/ | +10 | 2 replies | Applied |
| 2/13 | mm: rename vma_flag_test/set_atomic() to vma_test/set_atomic_flag()include/ mm/ mm/ | +7 | 2 replies | Applied |
| 3/13 | mm: add mk_vma_flags() bitmap flag macro helperinclude/ | +33 | 3 replies | Applied |
| 4/13 | tools: bitmap: add missing bitmap_[subset(), andnot()]tools/ tools/ | +51 | 1 reply | Applied |
| 5/13 | mm: add basic VMA flag operation helper functionsinclude/ include/ tools/ | +295 | 3 replies | Applied |
| 6/13 | mm: update hugetlbfs to use VMA flags on mmap_preparefs/ include/ include/ +4 more | +41 | 1 reply | Applied |
| 7/13 | mm: update secretmem to use VMA flags on mmap_preparemm/ mm/ mm/ +2 more | +10 | 4 replies | |
| 8/13 | mm: update shmem_[kernel]_file_*() functions to use vma_flags_tarch/ drivers/ drivers/ +13 more | +57 | 7 replies | Applied |
| 9/13 | mm: update all remaining mmap_prepare users to use vma_flags_tdrivers/ drivers/ fs/ +13 more | +56 | 6 replies | Applied |
| 10/13 | mm: make vm_area_desc utilise vma_flags_t onlyinclude/ include/ mm/ +4 more | +25 | 2 replies | Applied |
| 11/13 | tools/testing/vma: separate VMA userland tests into separate filestools/ tools/ tools/ +5 more | +406 | 2 replies | Applied |
| 12/13 | tools/testing/vma: separate out vma_internal.h into logical headerstools/ tools/ tools/ +2 more | +1885 | 2 replies | |
| 13/13 | tools/testing/vma: add VMA userland tests for VMA flag functionstools/ tools/ tools/ +2 more | +322 | 1 reply | Applied |
Reviews trailers per patch
All 13 patches have a Reviewed-by or Acked-by.
| Patch | Damien Le Moal | Baolin Wang | Darrick J. Wong | Jarkko Sakkinen | Pedro Falcato | Liam R. Howlett |
|---|---|---|---|---|---|---|
| 1/13 | R-b | R-b | ||||
| 2/13 | R-b | R-b | ||||
| 3/13 | R-b | R-b | ||||
| 4/13 | A-b | |||||
| 5/13 | R-b | R-b | ||||
| 6/13 | R-b | |||||
| 7/13 | R-b | |||||
| 8/13 | R-b | R-b | R-b | |||
| 9/13 | A-b | A-b | A-b | |||
| 10/13 | R-b | R-b | ||||
| 11/13 | R-b | |||||
| 12/13 | R-b | |||||
| 13/13 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 6 reviewers.
Cover letter [PATCH v2 00/13] mm: add bitmap VMA flag helpers and convert all mmap_prepare to use them
We introduced the bitmap VMA type vma_flags_t in the aptly named commit 9ea35a25d51b ("mm: introduce VMA flags bitmap type") in order to permit future growth in VMA flags and to prevent the asinine requirement that VMA flags be available to 64-bit kernels only if they happened to use a bit number about 32-bits.
This is a long-term project as there are very many users of VMA flags within the kernel that need to be updated in order to utilise this new type.
In order to further this aim, this series adds a number of helper functions to enable ordinary interactions with VMA flags - that is testing, setting and clearing them.
In order to make working with VMA bit numbers less cumbersome this series introduces the mk_vma_flags() helper macro which generates a vma_flags_t from a variadic parameter list, e.g.:
vma_flags_t flags = mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT, VMA_EXEC_BIT);
It turns out that the compiler optimises this very well to the point that this is just as efficient as using VM_xxx pre-computed bitmap values.
This series then introduces the following functions:
bool vma_flags_test_mask(vma_flags_t flags, vma_flags_t to_test); bool vma_flags_test_all_mask(vma_flags_t flags, vma_flags_t to_test); void vma_flags_set_mask(vma_flags_t *flags, vma_flags_t to_set); void vma_flags_clear_mask(vma_flags_t *flags, vma_flags_t to_clear);
Providing means of testing any flag, testing all flags, setting, and clearing a specific vma_flags_t mask.
For convenience, helper macros are provided - vma_flags_test(), vma_flags_set() and vma_flags_clear(), each of which utilise mk_vma_flags() to make these operations easier, as well as an EMPTY_VMA_FLAGS macro to make initialisation of an empty vma_flags_t value easier, e.g.:
vma_flags_t flags = EMPTY_VMA_FLAGS;
vma_flags_set(&flags, VMA_READ_BIT, VMA_WRITE_BIT, VMA_EXEC_BIT);
...
if (vma_flags_test(flags, VMA_READ_BIT)) {
...
}
...
if (vma_flags_test_all_mask(flags, VMA_REMAP_FLAGS)) {
...
}
...
vma_flags_clear(&flags, VMA_READ_BIT);Since callers are often dealing with a vm_area_struct (VMA) or vm_area_desc (VMA descriptor as used in .mmap_prepare) object, this series further provides helpers for these - firstly vma_set_flags_mask() and vma_set_flags() for a VMA:
vma_flags_t flags = EMPTY_VMA_FLAGS: vma_flags_set(&flags, VMA_READ_BIT, VMA_WRITE_BIT, VMA_EXEC_BIT); ... vma_set_flags_mask(&vma, flags); ... vma_set_flags(&vma, VMA_DONTDUMP_BIT);
Note that these do NOT ensure appropriate locks are taken and assume the callers takes care of this.
For VMA descriptors this series adds vma_desc_[test, set, clear]_flags_mask() and vma_desc_[test, set, clear]_flags() for a VMA descriptor, e.g.:
static int foo_mmap_prepare(struct vm_area_desc *desc)
{
...
vma_desc_set_flags(desc, VMA_SEQ_READ_BIT);
vma_desc_clear_flags(desc, VMA_RAND_READ_BIT);
...
if (vma_desc_test_flags(desc, VMA_SHARED_BIT) {
...
}
...
}With these helpers introduced, this series then updates all mmap_prepare users to make use of the vma_flags_t vm_area_desc->vma_flags field rather than the legacy vm_flags_t vm_area_desc->vm_flags field.
In order to do so, several other related functions need to be updated, with separate patches for larger changes in hugetlbfs, secretmem and shmem before finally removing vm_area_desc->vm_flags altogether.
This lays the foundations for future elimination of vm_flags_t and associated defines and functionality altogether in the long run, and elimination of the use of vm_flags_t in f_op->mmap() hooks in the near term as mmap_prepare replaces these.
There is a useful synergy between the VMA flags and mmap_prepare work here as with this change in place, converting f_op->mmap() to f_op->mmap_prepare naturally also converts use of vm_flags_t to vma_flags_t in all drivers which declare mmap handlers.
This accounts for the majority of the users of the legacy vm_flags_*() helpers and thus a large number of drivers which need to interact with VMA flags in general.
This series also updates the userland VMA tests to account for the change, and adds unit tests for these helper functions to assert that they behave as expected.
In order to faciliate this change in a sensible way, the series also separates out the VMA unit tests into - code that is duplicated from the kernel that should be kept in sync, code that is customised for test purposes and code that is stubbed out.
We also separate out the VMA userland tests into separate files to make it easier to manage and to provide a sensible baseline for adding the userland tests for these helpers.
REVIEWS NOTE: I rebased this on
https://lore.kernel.org/linux-mm/[email protected]/
in order to make life easier with conflict resolutions.
v2:
* Rebased on mm-unstable again, and then on my other series :)
* Removed sparse __private decoration - it is not doing us any good, and anybody accessing vma->flags.__vma_flags will be very obviously doing something wrong (TM) so we don't really need it. This will shut up the various bots reporting sparse issues :)
* Made functions which test VMA flags reference a const vma_flags_t * pointer for the flags being checked as per Jason.
* Updated entire series to affect this change, including userland tests.
* Fixed missed fixup in memfd_luo_retrieve() and folded into series. Turns out this doesn't get built with allmodconfig due to my arch not support.
* Fixed up issue with dev dax reported by Zi (thanks, much appreciated!)
v1 resend:
* Rebased on mm-unstable to fix vma_internal.h conflict tested and confirmed working.
https://lore.kernel.org/all/[email protected]/
v1:
https://lore.kernel.org/all/[email protected]/
Lorenzo Stoakes (13):
mm/vma: remove __private sparse decoration from vma_flags_t
mm: rename vma_flag_test/set_atomic() to vma_test/set_atomic_flag()
mm: add mk_vma_flags() bitmap flag macro helper
tools: bitmap: add missing bitmap_[subset(), andnot()]
mm: add basic VMA flag operation helper functions
mm: update hugetlbfs to use VMA flags on mmap_prepare
mm: update secretmem to use VMA flags on mmap_prepare
mm: update shmem_[kernel]_file_*() functions to use vma_flags_t
mm: update all remaining mmap_prepare users to use vma_flags_t
mm: make vm_area_desc utilise vma_flags_t only tools/testing/vma: separate VMA userland tests into separate files tools/testing/vma: separate out vma_internal.h into logical headers tools/testing/vma: add VMA userland tests for VMA flag functions
arch/x86/kernel/cpu/sgx/ioctl.c | 2 +-
drivers/char/mem.c | 6 +-
drivers/dax/device.c | 10 +-
drivers/gpu/drm/drm_gem.c | 5 +-
drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 2 +-
drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 3 +-
drivers/gpu/drm/i915/gt/shmem_utils.c | 3 +-
drivers/gpu/drm/ttm/tests/ttm_tt_test.c | 2 +-
drivers/gpu/drm/ttm/ttm_backup.c | 3 +-
drivers/gpu/drm/ttm/ttm_tt.c | 2 +-
fs/aio.c | 2 +-
fs/erofs/data.c | 5 +-
fs/ext4/file.c | 4 +-
fs/hugetlbfs/inode.c | 14 +-
fs/ntfs3/file.c | 2 +-
fs/orangefs/file.c | 4 +-
fs/ramfs/file-nommu.c | 2 +-
fs/resctrl/pseudo_lock.c | 2 +-
fs/romfs/mmap-nommu.c | 2 +-
fs/xfs/scrub/xfile.c | 3 +-
fs/xfs/xfs_buf_mem.c | 2 +-
fs/xfs/xfs_file.c | 4 +-
fs/zonefs/file.c | 3 +-
include/linux/dax.h | 8 +-
include/linux/hugetlb.h | 6 +-
include/linux/hugetlb_inline.h | 10 +
include/linux/mm.h | 248 ++-
include/linux/mm_types.h | 23 +-
include/linux/shmem_fs.h | 8 +-
ipc/shm.c | 12 +-
kernel/relay.c | 2 +-
mm/filemap.c | 2 +-
mm/hugetlb.c | 22 +-
mm/internal.h | 2 +-
mm/khugepaged.c | 2 +-
mm/madvise.c | 2 +-
mm/memfd.c | 6 +-
mm/memfd_luo.c | 2 +-
mm/memory.c | 17 +-
mm/mmap.c | 10 +-
mm/mremap.c | 2 +-
mm/secretmem.c | 7 +-
mm/shmem.c | 59 +-
mm/util.c | 2 +-
mm/vma.c | 13 +-
mm/vma.h | 3 +-
security/keys/big_key.c | 2 +-
tools/include/linux/bitmap.h | 22 +
tools/lib/bitmap.c | 29 +
tools/testing/vma/Makefile | 7 +-
tools/testing/vma/include/custom.h | 119 ++
tools/testing/vma/include/dup.h | 1332 ++++++++++++++
tools/testing/vma/include/stubs.h | 428 +++++
tools/testing/vma/main.c | 55 +
tools/testing/vma/shared.c | 131 ++
tools/testing/vma/shared.h | 114 ++
tools/testing/vma/{vma.c => tests/merge.c} | 332 +---
tools/testing/vma/tests/mmap.c | 57 +
tools/testing/vma/tests/vma.c | 339 ++++
tools/testing/vma/vma_internal.h | 1847 +-------------------
60 files changed, 3055 insertions(+), 2314 deletions(-)
create mode 100644 tools/testing/vma/include/custom.h
create mode 100644 tools/testing/vma/include/dup.h
create mode 100644 tools/testing/vma/include/stubs.h
create mode 100644 tools/testing/vma/main.c
create mode 100644 tools/testing/vma/shared.c
create mode 100644 tools/testing/vma/shared.h
rename tools/testing/vma/{vma.c => tests/merge.c} (82%)
create mode 100644 tools/testing/vma/tests/mmap.c
create mode 100644 tools/testing/vma/tests/vma.c--
2.52.0