Add pseudo-anonymous huge page mappings V3
Subsystems (10)
cover Add pseudo-anonymous huge page mappings V3
This patch set adds a flag to mmap that allows the user to request a mapping to be backed with huge pages. This mapping will borrow functionality from the huge page shm code to create a file on the kernel internal mount and use it to approximate an anonymous mapping. The MAP_HUGETLB flag is a modifier to MAP_ANONYMOUS and will not work without both flags being preset.
A new flag is necessary because there is no other way to hook into huge pages without creating a file on a hugetlbfs mount which wouldn't be MAP_ANONYMOUS.
To userspace, this mapping will behave just like an anonymous mapping because the file is not accessible outside of the kernel.
This patch set is meant to simplify the programming model, presently there is a large chunk of boiler plate code, contained in libhugetlbfs, required to create private, hugepage backed mappings. This patch set would allow use of hugepages without linking to libhugetlbfs or having hugetblfs mounted.
Unification of the VM code would provide these same benefits, but it has been resisted each time that it has been suggested for several
reasons: it would break PAGE_SIZE assumptions across the kernel, it makes page-table abstractions really expensive, and it does not provide any benefit on architectures that do not support huge pages, incurring fast path penalties without providing any benefit on these architectures.
This verion includes the fixes posted to linux-mm as well as additions to mman.h for the four archtiectures that do not make use of mman-common.h. The addition of the MAP_HUGETLB flag to these four (xtensa, parisc, alpha, and mips) is required because MAP_HUGETLB is used in common vm code.
Eric B Munson (7):
hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on
the vfs internal mount
Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions Add MAP_HUGETLB example
Add MAP_HUGETLB flag to alpha mman.h
Add MAP_HUGETLB flag to xtensa mman.h
Add MAP_HUGETLB flag to parisc mman.h
Add MAP_HUGETLB flag to mips mman.h
Documentation/vm/00-INDEX | 2 + Documentation/vm/hugetlbpage.txt | 14 ++++--- Documentation/vm/map_hugetlb.c | 77 +++++++++++++++++++++++++++++++++++++ arch/alpha/include/asm/mman.h | 6 +++ arch/mips/include/asm/mman.h | 6 +++ arch/parisc/include/asm/mman.h | 7 +++ arch/xtensa/include/asm/mman.h | 6 +++ fs/hugetlbfs/inode.c | 13 +++++- include/asm-generic/mman-common.h | 1 + include/linux/hugetlb.h | 19 ++++++++- ipc/shm.c | 2 +- mm/mmap.c | 19 +++++++++ 12 files changed, 160 insertions(+), 12 deletions(-) create mode 100644 Documentation/vm/map_hugetlb.c
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 1/7 hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on the vfs internal mount
There are two means of creating mappings backed by huge pages:
1. mmap() a file created on hugetlbfs
2. Use shm which creates a file on an internal mount which essentially maps it MAP_SHARED
The internal mount is only used for shared mappings but there is very little that stops it being used for private mappings. This patch extends hugetlbfs_file_setup() to deal with the creation of files that will be mapped MAP_PRIVATE on the internal hugetlbfs mount. This extended API is used in a subsequent patch to implement the MAP_HUGETLB mmap() flag.
--- fs/hugetlbfs/inode.c | 13 ++++++++++--- include/linux/hugetlb.h | 12 ++++++++++-- ipc/shm.c | 2 +- 3 files changed, 21 insertions(+), 6 deletions(-)
fs/hugetlbfs/inode.c +10 -3
@@ -507,6 +507,13 @@ static struct inode *hugetlbfs_get_inode(struct super_block *sb, uid_t uid, inode->i_atime = inode->i_mtime = inode->i_ctime =CURRENT_TIME ;INIT_LIST_HEAD (&inode->i_mapping->private_list); info =HUGETLBFS_I (inode); +/* +* The policy is initialized here even if we are creating a +* private inode because initialization simply creates an +* an empty rb tree and calls spin_lock_init(), later when we +* call mpol_free_shared_policy() it will just return because +* the rb tree will still be empty. +*/ mpol_shared_policy_init (&info->policy,NULL );switch (mode &S_IFMT ) {default :
@@ -931,13 +938,13 @@ static struct file_system_type hugetlbfs_fs_type = {static struct vfsmount *hugetlbfs_vfsmount; -static int can_do_hugetlb_shm (void ) +static int can_do_hugetlb_shm () {return capable (CAP_IPC_LOCK ) ||in_group_p (sysctl_hugetlb_shm_group); }struct file *hugetlb_file_setup (const char *name,size_t size,int acctflag, -struct user_struct **user) +struct user_struct **user,int creat_flags) {int error = -ENOMEM ;struct file *file;
@@ -949,7 +956,7 @@ struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag,if (!hugetlbfs_vfsmount)return ERR_PTR (-ENOENT ); -if (!can_do_hugetlb_shm ()) { +if (creat_flags ==HUGETLB_SHMFS_INODE && !can_do_hugetlb_shm ()) { *user =current_user ();if (user_shm_lock (size, *user)) { WARN_ONCE(1,
include/linux/hugetlb.h +10 -2
@@ -110,6 +110,14 @@ static inline void hugetlb_report_meminfo(struct seq_file *m)#endif /* !CONFIG_HUGETLB_PAGE */ +enum { +/* +* The file will be used as an shm file so shmfs accounting rules +* apply +*/ +HUGETLB_SHMFS_INODE =1 , +}; + #ifdef CONFIG_HUGETLBFS struct hugetlbfs_config { uid_t uid;
@@ -148,7 +156,7 @@ static inline struct hugetlbfs_sb_info *HUGETLBFS_SB(struct super_block *sb)extern const struct file_operations hugetlbfs_file_operations;extern struct vm_operations_struct hugetlb_vm_ops;struct file *hugetlb_file_setup (const char *name,size_t size,int acct, -struct user_struct **user); +struct user_struct **user,int creat_flags);int hugetlb_get_quota (struct address_space *mapping,long delta);void hugetlb_put_quota (struct address_space *mapping,long delta);
@@ -170,7 +178,7 @@ static inline void set_file_hugepages(struct file *file)#define is_file_hugepages (file) 0#define set_file_hugepages (file) BUG() -#define hugetlb_file_setup (name,size,acct,user) ERR_PTR(-ENOSYS) +#define hugetlb_file_setup (name,size,acct,user,creat) ERR_PTR(-ENOSYS)#endif /* !CONFIG_HUGETLBFS */
ipc/shm.c +1 -1
@@ -370,7 +370,7 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)if (shmflg &SHM_NORESERVE ) acctflag =VM_NORESERVE ; file =hugetlb_file_setup (name, size, acctflag, - &shp->mlock_user); + &shp->mlock_user,HUGETLB_SHMFS_INODE ); }else {/* * Do not allow no accounting for OVERCOMMIT_NEVER, even
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 2/7 Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions
This patch adds a flag for mmap that will be used to request a huge page region that will look like anonymous memory to user space. This is accomplished by using a file on the internal vfsmount. MAP_HUGETLB is a modifier of MAP_ANONYMOUS and so must be specified with it. The region will behave the same as a MAP_ANONYMOUS region using small pages.
--- include/asm-generic/mman-common.h | 1 + include/linux/hugetlb.h | 7 +++++++ mm/mmap.c | 19 +++++++++++++++++++ 3 files changed, 27 insertions(+), 0 deletions(-)
include/asm-generic/mman-common.h +1 -0
@@ -19,6 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#define MAP_HUGETLB 0x080000/* create a huge page mapping */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
include/linux/hugetlb.h +7 -0
@@ -110,12 +110,19 @@ static inline void hugetlb_report_meminfo(struct seq_file *m)#endif /* !CONFIG_HUGETLB_PAGE */ +#define HUGETLB_ANON_FILE "anon_hugepage" +enum {/* * The file will be used as an shm file so shmfs accounting rules * apply */ HUGETLB_SHMFS_INODE =1 , +/* +* The file is being created on the internal vfs mount and shmfs +* accounting rules do not apply +*/ +HUGETLB_ANONHUGE_INODE =2 , };#ifdef CONFIG_HUGETLBFS
mm/mmap.c +19 -0
@@ -29,6 +29,7 @@#include <linux/rmap.h> #include <linux/mmu_notifier.h> #include <linux/perf_counter.h> +#include <linux/hugetlb.h> #include <asm/uaccess.h> #include <asm/cacheflush.h>
@@ -951,6 +952,24 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,if (mm->map_count > sysctl_max_map_count)return -ENOMEM ; +if (flags &MAP_HUGETLB ) { +struct user_struct *user =NULL ; +if (file) +return -EINVAL ; + +/* +* VM_NORESERVE is used because the reservations will be +* taken when vm_ops->mmap() is called +* A dummy user value is used because we are not locking +* memory so no accounting is necessary +*/ + len =ALIGN (len,huge_page_size (&default_hstate)); + file =hugetlb_file_setup (HUGETLB_ANON_FILE , len,VM_NORESERVE , + &user,HUGETLB_ANONHUGE_INODE ); +if (IS_ERR (file)) +return PTR_ERR (file); + } +/* Obtain the address to map to. we verify (or select) it and ensure * that it represents a valid section of the address space. */
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 3/7 Add MAP_HUGETLB example
This patch adds an example of how to use the MAP_HUGETLB flag to the vm documentation directory and a reference to the example in hugetlbpage.txt.
--- Documentation/vm/00-INDEX | 2 + Documentation/vm/hugetlbpage.txt | 14 ++++--- Documentation/vm/map_hugetlb.c | 77 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+), 6 deletions(-) create mode 100644 Documentation/vm/map_hugetlb.c
Documentation/vm/00-INDEX +2 -0
@@ -20,3 +20,5 @@ slabinfo.c - source code for a tool to get reports about slabs. slub.txt - a short users guide for SLUB. +map_hugetlb.c + - an example program that uses the MAP_HUGETLB mmap flag.
Documentation/vm/hugetlbpage.txt +8 -6
@@ -146,12 +146,14 @@ Regular chown, chgrp, and chmod commands (with right permissions) could be used to change the file attributes on hugetlbfs. Also, it is important to note that no such mount command is required if the -applications are going to use only shmat/shmget system calls. Users who -wish to use hugetlb page via shared memory segment should be a member of -a supplementary group and system admin needs to configure that gid into -/proc/sys/vm/hugetlb_shm_group. It is possible for same or different -applications to use any combination of mmaps and shm* calls, though the -mount of filesystem will be required for using mmap calls. +applications are going to use only shmat/shmget system calls or mmap with +MAP_HUGETLB. Users who wish to use hugetlb page via shared memory segment +should be a member of a supplementary group and system admin needs to +configure that gid into /proc/sys/vm/hugetlb_shm_group. It is possible for +same or different applications to use any combination of mmaps and shm* +calls, though the mount of filesystem will be required for using mmap calls +without MAP_HUGETLB. For an example of how to use mmap with MAP_HUGETLB see +map_hugetlb.c. *******************************************************************
Documentation/vm/map_hugetlb.c new file +77 -0
77 lines not on this page · raw message
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 4/7 Add MAP_HUGETLB flag to alpha mman.h
Even though alpha does not support huge pages this flag needs to be defined here to keep the compiler happy. This is because alpha does not make use of mman-common.h, so any flags defined there and used in common code need to be added to alpha's mman.h manually.
--- arch/alpha/include/asm/mman.h | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
arch/alpha/include/asm/mman.h +6 -0
@@ -28,6 +28,12 @@#define MAP_NORESERVE 0x10000/* don't check for reservations */ #define MAP_POPULATE 0x20000/* populate (prefault) pagetables */ #define MAP_NONBLOCK 0x40000/* do not block on IO */ +/* +* This flag is included even though huge pages are not supported because +* the flag is defined in mman-common.h and used in common vm code but +* mman-common.h is not included here +*/ +#define MAP_HUGETLB 0x080000#define MS_ASYNC 1/* sync memory asynchronously */ #define MS_SYNC 2/* synchronous memory sync */
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 5/7 Add MAP_HUGETLB flag to xtensa mman.h
Even though xtensa does not support huge pages this flag needs to be defined here to keep the compiler happy. This is because xtensa does not make use of mman-common.h, so any flags defined there and used in common code need to be added to xtensa's mman.h manually.
--- arch/xtensa/include/asm/mman.h | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
arch/xtensa/include/asm/mman.h +6 -0
@@ -53,6 +53,12 @@#define MAP_LOCKED 0x8000/* pages are locked */ #define MAP_POPULATE 0x10000/* populate (prefault) pagetables */ #define MAP_NONBLOCK 0x20000/* do not block on IO */ +/* +* This flag is included even though huge pages are not supported because +* the flag is defined in mman-common.h and used in common vm code but +* mman-common.h is not included here +*/ +#define MAP_HUGETLB 0x080000/* * Flags for msync
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 6/7 Add MAP_HUGETLB flag to parisc mman.h
Even though parisc does not support huge pages this flag needs to be defined here to keep the compiler happy. This is because parisc does not make use of mman-common.h, so any flags defined there and used in common code need to be added to parisc's mman.h manually.
--- arch/parisc/include/asm/mman.h | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-)
arch/parisc/include/asm/mman.h +7 -0
@@ -58,4 +58,11 @@#define MAP_FILE 0#define MAP_VARIABLE 0 +/* +* This flag is included even though huge pages are not supported because +* the flag is defined in mman-common.h and used in common vm code but +* mman-common.h is not included here +*/ +#define MAP_HUGETLB 0x080000 +#endif /* __PARISC_MMAN_H__ */
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 7/7 Add MAP_HUGETLB flag to mips mman.h
Even though mips does not support huge pages this flag needs to be defined here to keep the compiler happy. This is because mips does not make use of mman-common.h, so any flags defined there and used in common code need to be added to mips's mman.h manually.
--- arch/mips/include/asm/mman.h | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
arch/mips/include/asm/mman.h +6 -0
@@ -46,6 +46,12 @@#define MAP_LOCKED 0x8000/* pages are locked */ #define MAP_POPULATE 0x10000/* populate (prefault) pagetables */ #define MAP_NONBLOCK 0x20000/* do not block on IO */ +/* +* This flag is included even though huge pages are not supported because +* the flag is defined in mman-common.h and used in common vm code but +* mman-common.h is not included here +*/ +#define MAP_HUGETLB 0x080000/* * Flags for msync
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 1/7 hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on the vfs internal mount
In reply to Eric B MunsonThere are two means of creating mappings backed by huge pages:
1. mmap() a file created on hugetlbfs
2. Use shm which creates a file on an internal mount which essentially maps it MAP_SHARED
The internal mount is only used for shared mappings but there is very little that stops it being used for private mappings. This patch extends hugetlbfs_file_setup() to deal with the creation of files that will be mapped MAP_PRIVATE on the internal hugetlbfs mount. This extended API is used in a subsequent patch to implement the MAP_HUGETLB mmap() flag.
--- fs/hugetlbfs/inode.c | 11 +++++++++-- include/linux/hugetlb.h | 12 ++++++++++-- ipc/shm.c | 2 +- 3 files changed, 20 insertions(+), 5 deletions(-)
fs/hugetlbfs/inode.c +9 -2
@@ -507,6 +507,13 @@ static struct inode *hugetlbfs_get_inode(struct super_block *sb, uid_t uid, inode->i_atime = inode->i_mtime = inode->i_ctime =CURRENT_TIME ;INIT_LIST_HEAD (&inode->i_mapping->private_list); info =HUGETLBFS_I (inode); +/* +* The policy is initialized here even if we are creating a +* private inode because initialization simply creates an +* an empty rb tree and calls spin_lock_init(), later when we +* call mpol_free_shared_policy() it will just return because +* the rb tree will still be empty. +*/ mpol_shared_policy_init (&info->policy,NULL );switch (mode &S_IFMT ) {default :
@@ -937,7 +944,7 @@ static int can_do_hugetlb_shm(void) }struct file *hugetlb_file_setup (const char *name,size_t size,int acctflag, -struct user_struct **user) +struct user_struct **user,int creat_flags) {int error = -ENOMEM ;struct file *file;
@@ -949,7 +956,7 @@ struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag,if (!hugetlbfs_vfsmount)return ERR_PTR (-ENOENT ); -if (!can_do_hugetlb_shm ()) { +if (creat_flags ==HUGETLB_SHMFS_INODE && !can_do_hugetlb_shm ()) { *user =current_user ();if (user_shm_lock (size, *user)) { WARN_ONCE(1,
include/linux/hugetlb.h +10 -2
@@ -110,6 +110,14 @@ static inline void hugetlb_report_meminfo(struct seq_file *m)#endif /* !CONFIG_HUGETLB_PAGE */ +enum { +/* +* The file will be used as an shm file so shmfs accounting rules +* apply +*/ +HUGETLB_SHMFS_INODE =1 , +}; + #ifdef CONFIG_HUGETLBFS struct hugetlbfs_config { uid_t uid;
@@ -148,7 +156,7 @@ static inline struct hugetlbfs_sb_info *HUGETLBFS_SB(struct super_block *sb)extern const struct file_operations hugetlbfs_file_operations;extern struct vm_operations_struct hugetlb_vm_ops;struct file *hugetlb_file_setup (const char *name,size_t size,int acct, -struct user_struct **user); +struct user_struct **user,int creat_flags);int hugetlb_get_quota (struct address_space *mapping,long delta);void hugetlb_put_quota (struct address_space *mapping,long delta);
@@ -170,7 +178,7 @@ static inline void set_file_hugepages(struct file *file)#define is_file_hugepages (file) 0#define set_file_hugepages (file) BUG() -#define hugetlb_file_setup (name,size,acct,user) ERR_PTR(-ENOSYS) +#define hugetlb_file_setup (name,size,acct,user,creat) ERR_PTR(-ENOSYS)#endif /* !CONFIG_HUGETLBFS */
ipc/shm.c +1 -1
@@ -370,7 +370,7 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)if (shmflg &SHM_NORESERVE ) acctflag =VM_NORESERVE ; file =hugetlb_file_setup (name, size, acctflag, - &shp->mlock_user); + &shp->mlock_user,HUGETLB_SHMFS_INODE ); }else {/* * Do not allow no accounting for OVERCOMMIT_NEVER, even
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 1/7] hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on the vfs internal mount
Quoted 6 lines · Eric B Munson
There are two means of creating mappings backed by huge pages:
1.
mmap()a file created on hugetlbfs
2. Use shm which creates a file on an internal mount which essentially maps itMAP_SHARED
Quoted 7 lines
The internal mount is only used for shared mappings but there is very little that stops it being used for private mappings. This patch extends
hugetlbfs_file_setup()to deal with the creation of files that will be mappedMAP_PRIVATEon the internal hugetlbfs mount. This extended API is used in a subsequent patch to implement theMAP_HUGETLBmmap()flag.Signed-off-by Eric Munson <[email protected]>
Re: [PATCH 2/7] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions
In reply to Eric B MunsonQuoted 7 lines · Eric B Munson
This patch adds a flag for mmap that will be used to request a huge page region that will look like anonymous memory to user space. This is accomplished by using a file on the internal vfsmount.
MAP_HUGETLBis a modifier ofMAP_ANONYMOUSand so must be specified with it. The region will behave the same as aMAP_ANONYMOUSregion using small pages.Signed-off-by Eric B Munson <[email protected]>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>