Add pseudo-anonymous huge page mappings V4
Subsystems (6)
cover Add pseudo-anonymous huge page mappings V4
This patch set adds a flag to mmap that allows the user to request a mapping to be backed with huge pages. This mapping will borrow functionality from the huge page shm code to create a file on the kernel internal mount and use it to approximate an anonymous mapping. The MAP_HUGETLB flag is a modifier to MAP_ANONYMOUS and will not work without both flags being preset.
A new flag is necessary because there is no other way to hook into huge pages without creating a file on a hugetlbfs mount which wouldn't be MAP_ANONYMOUS.
To userspace, this mapping will behave just like an anonymous mapping because the file is not accessible outside of the kernel.
This patch set is meant to simplify the programming model, presently there is a large chunk of boiler plate code, contained in libhugetlbfs, required to create private, hugepage backed mappings. This patch set would allow use of hugepages without linking to libhugetlbfs or having hugetblfs mounted.
Unification of the VM code would provide these same benefits, but it has been resisted each time that it has been suggested for several
reasons: it would break PAGE_SIZE assumptions across the kernel, it makes page-table abstractions really expensive, and it does not provide any benefit on architectures that do not support huge pages, incurring fast path penalties without providing any benefit on these architectures.
Eric B Munson (3):
hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on
the vfs internal mount
Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions Add MAP_HUGETLB example
Documentation/vm/00-INDEX | 2 + Documentation/vm/hugetlbpage.txt | 14 ++++--- Documentation/vm/map_hugetlb.c | 77 +++++++++++++++++++++++++++++++++++++ fs/hugetlbfs/inode.c | 21 ++++++++-- include/asm-generic/mman-common.h | 1 + include/linux/hugetlb.h | 19 ++++++++- ipc/shm.c | 2 +- mm/mmap.c | 19 +++++++++ 8 files changed, 142 insertions(+), 13 deletions(-) create mode 100644 Documentation/vm/map_hugetlb.c
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 1/3 hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on the vfs internal mount
There are two means of creating mappings backed by huge pages:
1. mmap() a file created on hugetlbfs
2. Use shm which creates a file on an internal mount which essentially maps it MAP_SHARED
The internal mount is only used for shared mappings but there is very little that stops it being used for private mappings. This patch extends hugetlbfs_file_setup() to deal with the creation of files that will be mapped MAP_PRIVATE on the internal hugetlbfs mount. This extended API is used in a subsequent patch to implement the MAP_HUGETLB mmap() flag.
--- fs/hugetlbfs/inode.c | 21 +++++++++++++++++---- include/linux/hugetlb.h | 12 ++++++++++-- ipc/shm.c | 2 +- 3 files changed, 28 insertions(+), 7 deletions(-)
fs/hugetlbfs/inode.c +17 -4
@@ -506,6 +506,13 @@ static struct inode *hugetlbfs_get_inode(struct super_block *sb, uid_t uid, inode->i_atime = inode->i_mtime = inode->i_ctime =CURRENT_TIME ;INIT_LIST_HEAD (&inode->i_mapping->private_list); info =HUGETLBFS_I (inode); +/* +* The policy is initialized here even if we are creating a +* private inode because initialization simply creates an +* an empty rb tree and calls spin_lock_init(), later when we +* call mpol_free_shared_policy() it will just return because +* the rb tree will still be empty. +*/ mpol_shared_policy_init (&info->policy,NULL );switch (mode &S_IFMT ) {default :
@@ -930,13 +937,19 @@ static struct file_system_type hugetlbfs_fs_type = {static struct vfsmount *hugetlbfs_vfsmount; -static int can_do_hugetlb_shm (void ) +static int can_do_hugetlb_shm (int creat_flags) { -return capable (CAP_IPC_LOCK ) ||in_group_p (sysctl_hugetlb_shm_group); +if (creat_flags !=HUGETLB_SHMFS_INODE ) +return 0 ; +if (capable (CAP_IPC_LOCK )) +return 1 ; +if (in_group_p (sysctl_hugetlb_shm_group)) +return 1 ; +return 0 ; }struct file *hugetlb_file_setup (const char *name,size_t size,int acctflag, -struct user_struct **user) +struct user_struct **user,int creat_flags) {int error = -ENOMEM ;struct file *file;
@@ -948,7 +961,7 @@ struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag,if (!hugetlbfs_vfsmount)return ERR_PTR (-ENOENT ); -if (!can_do_hugetlb_shm ()) { +if (!can_do_hugetlb_shm (creat_flags)) { *user =current_user ();if (user_shm_lock (size, *user)) { WARN_ONCE(1,
include/linux/hugetlb.h +10 -2
@@ -110,6 +110,14 @@ static inline void hugetlb_report_meminfo(struct seq_file *m)#endif /* !CONFIG_HUGETLB_PAGE */ +enum { +/* +* The file will be used as an shm file so shmfs accounting rules +* apply +*/ +HUGETLB_SHMFS_INODE =1 , +}; + #ifdef CONFIG_HUGETLBFS struct hugetlbfs_config { uid_t uid;
@@ -148,7 +156,7 @@ static inline struct hugetlbfs_sb_info *HUGETLBFS_SB(struct super_block *sb)extern const struct file_operations hugetlbfs_file_operations;extern struct vm_operations_struct hugetlb_vm_ops;struct file *hugetlb_file_setup (const char *name,size_t size,int acct, -struct user_struct **user); +struct user_struct **user,int creat_flags);int hugetlb_get_quota (struct address_space *mapping,long delta);void hugetlb_put_quota (struct address_space *mapping,long delta);
@@ -170,7 +178,7 @@ static inline void set_file_hugepages(struct file *file)#define is_file_hugepages (file) 0#define set_file_hugepages (file) BUG() -#define hugetlb_file_setup (name,size,acct,user) ERR_PTR(-ENOSYS) +#define hugetlb_file_setup (name,size,acct,user,creat) ERR_PTR(-ENOSYS)#endif /* !CONFIG_HUGETLBFS */
ipc/shm.c +1 -1
@@ -370,7 +370,7 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)if (shmflg &SHM_NORESERVE ) acctflag =VM_NORESERVE ; file =hugetlb_file_setup (name, size, acctflag, - &shp->mlock_user); + &shp->mlock_user,HUGETLB_SHMFS_INODE ); }else {/* * Do not allow no accounting for OVERCOMMIT_NEVER, even
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 2/3 Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions
This patch adds a flag for mmap that will be used to request a huge page region that will look like anonymous memory to user space. This is accomplished by using a file on the internal vfsmount. MAP_HUGETLB is a modifier of MAP_ANONYMOUS and so must be specified with it. The region will behave the same as a MAP_ANONYMOUS region using small pages.
--- include/asm-generic/mman-common.h | 1 + include/linux/hugetlb.h | 7 +++++++ mm/mmap.c | 19 +++++++++++++++++++ 3 files changed, 27 insertions(+), 0 deletions(-)
include/asm-generic/mman-common.h +1 -0
@@ -19,6 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#define MAP_HUGETLB 0x40/* create a huge page mapping */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
include/linux/hugetlb.h +7 -0
@@ -110,12 +110,19 @@ static inline void hugetlb_report_meminfo(struct seq_file *m)#endif /* !CONFIG_HUGETLB_PAGE */ +#define HUGETLB_ANON_FILE "anon_hugepage" +enum {/* * The file will be used as an shm file so shmfs accounting rules * apply */ HUGETLB_SHMFS_INODE =1 , +/* +* The file is being created on the internal vfs mount and shmfs +* accounting rules do not apply +*/ +HUGETLB_ANONHUGE_INODE =2 , };#ifdef CONFIG_HUGETLBFS
mm/mmap.c +19 -0
@@ -29,6 +29,7 @@#include <linux/rmap.h> #include <linux/mmu_notifier.h> #include <linux/perf_counter.h> +#include <linux/hugetlb.h> #include <asm/uaccess.h> #include <asm/cacheflush.h>
@@ -951,6 +952,24 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,if (mm->map_count > sysctl_max_map_count)return -ENOMEM ; +if (flags &MAP_HUGETLB ) { +struct user_struct *user =NULL ; +if (file) +return -EINVAL ; + +/* +* VM_NORESERVE is used because the reservations will be +* taken when vm_ops->mmap() is called +* A dummy user value is used because we are not locking +* memory so no accounting is necessary +*/ + len =ALIGN (len,huge_page_size (&default_hstate)); + file =hugetlb_file_setup (HUGETLB_ANON_FILE , len,VM_NORESERVE , + &user,HUGETLB_ANONHUGE_INODE ); +if (IS_ERR (file)) +return PTR_ERR (file); + } +/* Obtain the address to map to. we verify (or select) it and ensure * that it represents a valid section of the address space. */
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 3/3 Add MAP_HUGETLB example
This patch adds an example of how to use the MAP_HUGETLB flag to the vm documentation directory and a reference to the example in hugetlbpage.txt.
--- Documentation/vm/00-INDEX | 2 + Documentation/vm/hugetlbpage.txt | 14 ++++--- Documentation/vm/map_hugetlb.c | 77 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 87 insertions(+), 6 deletions(-) create mode 100644 Documentation/vm/map_hugetlb.c
Documentation/vm/00-INDEX +2 -0
@@ -20,3 +20,5 @@ slabinfo.c - source code for a tool to get reports about slabs. slub.txt - a short users guide for SLUB. +map_hugetlb.c + - an example program that uses the MAP_HUGETLB mmap flag.
Documentation/vm/hugetlbpage.txt +8 -6
@@ -146,12 +146,14 @@ Regular chown, chgrp, and chmod commands (with right permissions) could be used to change the file attributes on hugetlbfs. Also, it is important to note that no such mount command is required if the -applications are going to use only shmat/shmget system calls. Users who -wish to use hugetlb page via shared memory segment should be a member of -a supplementary group and system admin needs to configure that gid into -/proc/sys/vm/hugetlb_shm_group. It is possible for same or different -applications to use any combination of mmaps and shm* calls, though the -mount of filesystem will be required for using mmap calls. +applications are going to use only shmat/shmget system calls or mmap with +MAP_HUGETLB. Users who wish to use hugetlb page via shared memory segment +should be a member of a supplementary group and system admin needs to +configure that gid into /proc/sys/vm/hugetlb_shm_group. It is possible for +same or different applications to use any combination of mmaps and shm* +calls, though the mount of filesystem will be required for using mmap calls +without MAP_HUGETLB. For an example of how to use mmap with MAP_HUGETLB see +map_hugetlb.c. *******************************************************************
Documentation/vm/map_hugetlb.c new file +77 -0
77 lines not on this page · raw message
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 2/3] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions
In reply to Eric B MunsonQuoted 18 lines · Eric B Munson
This patch adds a flag for mmap that will be used to request a huge page region that will look like anonymous memory to user space. This is accomplished by using a file on the internal vfsmount.
MAP_HUGETLBis a modifier ofMAP_ANONYMOUSand so must be specified with it. The region will behave the same as aMAP_ANONYMOUSregion using small pages.Signed-off-by Eric B Munson <[email protected]>--- include/asm-generic/mman-common.h | 1 + include/linux/hugetlb.h | 7 +++++++ mm/mmap.c | 19 +++++++++++++++++++ 3 files changed, 27 insertions(+), 0 deletions(-)include/asm-generic/mman-common.h +0
-0
include/asm-generic/mman-common.h line 25
Quoted 7 lines
include/asm-generic/mman-common.h +1
-0@@ -19,6 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#define MAP_HUGETLB 0x40/* create a huge page mapping */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
I'm afraid you can't put MAP_HUGETLB in mman-common.h: that is picked up by most or all architectures (which is of course what you wanted!) but conflicts with a definition in at least one of them. When I boot up mmotm on powerpc, I get a warning:
Using mlock ulimits for SHM_HUGETLB deprecated
------------[ cut here ]------------ Badness at fs/hugetlbfs/inode.c:941 NIP: c0000000001f3038 LR: c0000000001f3034 CTR: 0000000000000000 REGS: c0000000275d7960 TRAP: 0700 Not tainted (2.6.31-rc7-mm2) MSR: 9000000000029032 <EE,ME,CE,IR,DR> CR: 24000484 XER: 00000000
TASK = c000000029fa94a0[1321] 'console-kit-dae' THREAD: c0000000275d4000 CPU: 3
GPR00: c0000000001f3034 c0000000275d7be0 c00000000071a908 0000000000000032
GPR04: 0000000000000000 ffffffffffffffff ffffffffffffffff 0000000000000000
GPR08: c0000000297dc1d0 c0000000275d4000 d00008008247fa08 0000000000000000
GPR12: 0000000024000442 c00000000074ba00 000000000fedb9a4 000000001049cd18
GPR16: 00000000100365d0 00000000104a9100 000000000fefc350 00000000104a9098
GPR20: 00000000104a9160 000000000fefc238 0000000000000000 0000000000200000
GPR24: 0000000000000000 0000000001000000 c0000000275d7d20 0000000001000000
GPR28: c00000000058c738 ffffffffffffffb5 c0000000006a93d0 c000000000791400 NIP [c0000000001f3038] .hugetlb_file_setup+0xd0/0x254
LR [c0000000001f3034] .hugetlb_file_setup+0xcc/0x254
Call Trace: [c0000000275d7be0] [c0000000001f3034] .hugetlb_file_setup+0xcc/0x254 (unreliable) [c0000000275d7cb0] [c0000000000ee240] .do_mmap_pgoff+0x184/0x424 [c0000000275d7d80] [c00000000000a9c8] .sys_mmap+0xc4/0x13c [c0000000275d7e30] [c0000000000075ac] syscall_exit+0x0/0x40
Instruction dump:f89a0000 4bef7111 60000000 2c230000 41820034 e93e8018 80090014 2f800000 40fe0030 e87e80b0 4823ff09 60000000 <0fe00000> e93e8018 38000001 90090014
Which won't be coming from any use of MAP_HUGETLB, but presumably from something using MAP_NORESERVE, defined as 0x40 in arch/powerpc/include/asm/mman.h.
I think you have to put your #define MAP_HUGETLB into include/asm-generic/mman.h (seems used by only three architectures), and into the arch/whatever/include/asm/mman.h of each architecture which uses asm-generic/mman-common.h without asm-generic/mman.h.
Hugh
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 2/3] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions
Quoted 62 lines · Hugh Dickins
Quoted 18 lines · Eric B Munson
This patch adds a flag for mmap that will be used to request a huge page region that will look like anonymous memory to user space. This is accomplished by using a file on the internal vfsmount.
MAP_HUGETLBis a modifier ofMAP_ANONYMOUSand so must be specified with it. The region will behave the same as aMAP_ANONYMOUSregion using small pages.Signed-off-by Eric B Munson <[email protected]>--- include/asm-generic/mman-common.h | 1 + include/linux/hugetlb.h | 7 +++++++ mm/mmap.c | 19 +++++++++++++++++++ 3 files changed, 27 insertions(+), 0 deletions(-)include/asm-generic/mman-common.h +0
-0include/asm-generic/mman-common.h line 25
Quoted 7 lines
include/asm-generic/mman-common.h +1
-0@@ -19,6 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#define MAP_HUGETLB 0x40/* create a huge page mapping */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */ I'm afraid you can't put
MAP_HUGETLBinmman-common.h: that is picked up by most or all architectures (which is of course what you wanted!) but conflicts with a definition in at least one of them. When I boot up mmotm on powerpc, I get a warning:Using mlock ulimits for
SHM_HUGETLBdeprecated------------[ cut here ]------------ Badness at fs/hugetlbfs/inode.c:941 NIP: c0000000001f3038 LR: c0000000001f3034 CTR: 0000000000000000 REGS: c0000000275d7960 TRAP: 0700 Not tainted (2.6.31-rc7-mm2) MSR: 9000000000029032 <EE,ME,CE,IR,DR> CR: 24000484 XER: 00000000TASK = c000000029fa94a0[1321] 'console-kit-dae' THREAD:
c0000000275d4000CPU: 3
GPR00:c0000000001f3034c0000000275d7be0c00000000071a9080000000000000032
GPR04: 0000000000000000 ffffffffffffffff ffffffffffffffff 0000000000000000
GPR08:c0000000297dc1d0c0000000275d4000d00008008247fa080000000000000000
GPR12: 0000000024000442c00000000074ba00000000000fedb9a4000000001049cd18
GPR16:00000000100365d000000000104a9100000000000fefc35000000000104a9098
GPR20:00000000104a9160000000000fefc2380000000000000000 0000000000200000
GPR24: 0000000000000000 0000000001000000c0000000275d7d200000000001000000
GPR28:c00000000058c738ffffffffffffffb5c0000000006a93d0c000000000791400NIP [c0000000001f3038] .hugetlb_file_setup+0xd0/0x254
LR [c0000000001f3034] .hugetlb_file_setup+0xcc/0x254Call Trace: [c0000000275d7be0] [c0000000001f3034] .hugetlb_file_setup+0xcc/0x254 (unreliable) [c0000000275d7cb0] [c0000000000ee240] .do_mmap_pgoff+0x184/0x424 [c0000000275d7d80] [c00000000000a9c8] .sys_mmap+0xc4/0x13c [c0000000275d7e30] [c0000000000075ac] syscall_exit+0x0/0x40Instruction dump:
f89a00004bef7111600000002c23000041820034e93e8018800900142f80000040fe0030e87e80b04823ff0960000000 <0fe00000>e93e801838000001 90090014Which won't be coming from any use of
MAP_HUGETLB, but presumably from something usingMAP_NORESERVE, defined as0x40inarch/powerpc/include/asm/mman.h.
Quoted 7 lines
I think you have to put your #define
MAP_HUGETLBintoinclude/asm-generic/mman.h(seems used by only three architectures), and into thearch/whatever/include/asm/mman.hof each architecture which usesasm-generic/mman-common.hwithoutasm-generic/mman.h.Hugh
This problem is the same that Mel Gorman reported (and fixed) in response to patch 1 of this series. I have forwarded the patch that addresses this problem on, but it has not been picked up.
The bug is not where MAP_HUGETLB is defined, rather how the patch handled can_do_hugetlb_shm(). If MAP_HUGETLB was specified, can_do_hugetlb_shm() returned 0 forcing a call to user_shm_lock() which is responisble for the warning about SHM_HUGETLB and mlock ulimits. The fix is to check if the file is to be used for SHM_HUGETLB and if not, skip the calls to can_do_hugetlb_shm() and user_shm_lock().
Signature · 3 lines
Eric B Munson IBM Linux Technology Center [email protected]
Re: [PATCH 2/3] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions
Quoted 75 lines · Eric B Munson
Quoted 62 lines · Hugh Dickins
Quoted 18 lines · Eric B Munson
This patch adds a flag for mmap that will be used to request a huge page region that will look like anonymous memory to user space. This is accomplished by using a file on the internal vfsmount.
MAP_HUGETLBis a modifier ofMAP_ANONYMOUSand so must be specified with it. The region will behave the same as aMAP_ANONYMOUSregion using small pages.Signed-off-by Eric B Munson <ebmunson-r/[email protected]>--- include/asm-generic/mman-common.h | 1 + include/linux/hugetlb.h | 7 +++++++ mm/mmap.c | 19 +++++++++++++++++++ 3 files changed, 27 insertions(+), 0 deletions(-)include/asm-generic/mman-common.h +0
-0include/asm-generic/mman-common.h line 25
Quoted 7 lines
include/asm-generic/mman-common.h +1
-0@@ -19,6 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ +#define MAP_HUGETLB 0x40/* create a huge page mapping */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */ I'm afraid you can't put
MAP_HUGETLBinmman-common.h: that is picked up by most or all architectures (which is of course what you wanted!) but conflicts with a definition in at least one of them. When I boot up mmotm on powerpc, I get a warning:Using mlock ulimits for
SHM_HUGETLBdeprecated------------[ cut here ]------------ Badness at fs/hugetlbfs/inode.c:941 NIP: c0000000001f3038 LR: c0000000001f3034 CTR: 0000000000000000 REGS: c0000000275d7960 TRAP: 0700 Not tainted (2.6.31-rc7-mm2) MSR: 9000000000029032 <EE,ME,CE,IR,DR> CR: 24000484 XER: 00000000TASK = c000000029fa94a0[1321] 'console-kit-dae' THREAD:
c0000000275d4000CPU: 3
GPR00:c0000000001f3034c0000000275d7be0c00000000071a9080000000000000032
GPR04: 0000000000000000 ffffffffffffffff ffffffffffffffff 0000000000000000
GPR08:c0000000297dc1d0c0000000275d4000d00008008247fa080000000000000000
GPR12: 0000000024000442c00000000074ba00000000000fedb9a4000000001049cd18
GPR16:00000000100365d000000000104a9100000000000fefc35000000000104a9098
GPR20:00000000104a9160000000000fefc2380000000000000000 0000000000200000
GPR24: 0000000000000000 0000000001000000c0000000275d7d200000000001000000
GPR28:c00000000058c738ffffffffffffffb5c0000000006a93d0c000000000791400NIP [c0000000001f3038] .hugetlb_file_setup+0xd0/0x254
LR [c0000000001f3034] .hugetlb_file_setup+0xcc/0x254Call Trace: [c0000000275d7be0] [c0000000001f3034] .hugetlb_file_setup+0xcc/0x254 (unreliable) [c0000000275d7cb0] [c0000000000ee240] .do_mmap_pgoff+0x184/0x424 [c0000000275d7d80] [c00000000000a9c8] .sys_mmap+0xc4/0x13c [c0000000275d7e30] [c0000000000075ac] syscall_exit+0x0/0x40Instruction dump:
f89a00004bef7111600000002c23000041820034e93e8018800900142f80000040fe0030e87e80b04823ff0960000000 <0fe00000>e93e801838000001 90090014Which won't be coming from any use of
MAP_HUGETLB, but presumably from something usingMAP_NORESERVE, defined as0x40inarch/powerpc/include/asm/mman.h.Quoted 7 lines
I think you have to put your #define
MAP_HUGETLBintoinclude/asm-generic/mman.h(seems used by only three architectures), and into thearch/whatever/include/asm/mman.hof each architecture which usesasm-generic/mman-common.hwithoutasm-generic/mman.h.Hugh
This problem is the same that Mel Gorman reported (and fixed) in response to patch 1 of this series. I have forwarded the patch that addresses this problem on, but it has not been picked up.
Quoted 6 lines
The bug is not where
MAP_HUGETLBis defined, rather how the patch handledcan_do_hugetlb_shm(). IfMAP_HUGETLBwas specified,can_do_hugetlb_shm()returned 0 forcing a call touser_shm_lock()which is responisble for the warning aboutSHM_HUGETLBand mlock ulimits. The fix is to check if the file is to be used forSHM_HUGETLBand if not, skip the calls tocan_do_hugetlb_shm()anduser_shm_lock().
Sorry, no, I disagree.
I agree that the fs/hugetlbfs/inode.c:941 message and backtrace in themselves are symptoms of the can_do_hugetlb_shm() bug that Mel reported and fixed (I'm agreeing a little too readily, I've not actually studied that bug and fix, I'm taking it on trust).
But that does not explain how last year's openSUSE 11.1 userspace was trying for a MAP_HUGETLB mapping at startup on PowerPC (but not on x86), while you're only introducing MAP_HUGETLB now.
That is explained by you #defining MAP_HUGETLB in include/asm-generic/ mman-common.h to a number which is already being used for other MAP_s on some architectures. That's a separate bug which needs to be fixed by distributing the MAP_HUGETLB definition across various asm*/mman.h.
Hugh
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/3] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions
On Tue, 01 Sep 2009, Hugh Dickins wrote:
snip
Quoted 12 lines
Sorry, no, I disagree.
I agree that the
fs/hugetlbfs/inode.c:941message and backtrace in themselves are symptoms of thecan_do_hugetlb_shm()bug that Mel reported and fixed (I'm agreeing a little too readily, I've not actually studied that bug and fix, I'm taking it on trust).But that does not explain how last year's openSUSE 11.1 userspace was trying for a
MAP_HUGETLBmapping at startup on PowerPC (but not on x86), while you're only introducingMAP_HUGETLBnow.
Quoted 7 lines
That is explained by you #defining
MAP_HUGETLBininclude/asm-generic/mman-common.hto a number which is already being used for otherMAP_son some architectures. That's a separate bug which needs to be fixed by distributing theMAP_HUGETLBdefinition across variousasm*/mman.h.Hugh
Would it be okay to keep the define in include/asm-generic/mman.h if a value that is known free across all architectures is used? 0x080000 is not used by any arch and, AFAICT would work just as well.
Signature · 3 lines
Eric B Munson IBM Linux Technology Center [email protected]
Re: [PATCH 2/3] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions
Quoted 10 lines · Eric B Munson
Quoted 5 lines · Hugh Dickins
That is explained by you #defining
MAP_HUGETLBininclude/asm-generic/mman-common.hto a number which is already being used for otherMAP_son some architectures. That's a separate bug which needs to be fixed by distributing theMAP_HUGETLBdefinition across variousasm*/mman.h.Would it be okay to keep the define in
include/asm-generic/mman.hif a value that is known free across all architectures is used?0x080000is not used by any arch and, AFAICT would work just as well.
That's a very sensible suggestion, but departs from how we have assigned new numbers up until now: so include/asm-generic/mman-common.h isn't actually where we'd expect to find a Linux-specific MAP_ define.
I'd say, yes, do that for now, so as not to hit this conflict while testing in mmotm. But whether it should stay that way, or later the arch/*/include/asm/mman.h's be updated as I'd imagined, I don't know.
Arnd, Michael, do you have any views on this?
Thanks,
Hugh
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Arnd Bergmann
maintainerRe: [PATCH 2/3] Add MAP_HUGETLB for mmaping pseudo-anonymous huge page regions
Quoted 16 lines · Hugh Dickins
Quoted 10 lines · Eric B Munson
Quoted 5 lines · Hugh Dickins
That is explained by you #defining
MAP_HUGETLBininclude/asm-generic/mman-common.hto a number which is already being used for otherMAP_son some architectures. That's a separate bug which needs to be fixed by distributing theMAP_HUGETLBdefinition across variousasm*/mman.h.Would it be okay to keep the define in
include/asm-generic/mman.hif a value that is known free across all architectures is used?0x080000is not used by any arch and, AFAICT would work just as well.That's a very sensible suggestion, but departs from how we have assigned new numbers up until now: so
include/asm-generic/mman-common.hisn't actually where we'd expect to find a Linux-specificMAP_define.
Quoted 5 lines
I'd say, yes, do that for now, so as not to hit this conflict while testing in mmotm. But whether it should stay that way, or later the arch/*/include/asm/mman.h's be updated as I'd imagined, I don't know.
Arnd, Michael, do you have any views on this?
The minimal procedure would be to add it to mman-common.h, plus the asm/mman.h files for xtensa, mips, parisc and alpha, which all use a version that is compatible to a Unix variant, but that would be confusing the next person that needs to add a flag.
I'd use the number 0x40000 for all architectures except alpha, because that makes the most sense for asm-generic/mman.h. Alpha is weird anyway here, so we don't need to avoid conflicts with it.
With a few exceptions (sparc, powerpc), I think we should change all architectures to use asm-generic/mman.h instead of mman-common.h in the long run. If you touch those anyway, one option would be to do it in one step.
Arnd <><
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
PATCH 1/1 MAP_HUGETLB value collision fix
In reply to Hugh DickinsThe patch hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch used the value 0x40 for MAP_HUGETLB which is the same value used for various other flags on some architectures. This collision causes unexpected use of huge pages in the best case and mmap to fail with ENOMEM or ENOSYS in the worst. This patch changes the value for MAP_HUGETLB to a value that is not currently used on any arch.
This patch should be considered a fix to hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch.
--- include/asm-generic/mman-common.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
include/asm-generic/mman-common.h +1 -1
@@ -19,7 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ -#define MAP_HUGETLB 0x40/* create a huge page mapping */ +#define MAP_HUGETLB 0x080000/* create a huge page mapping */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 1/1 MAP_HUGETLB value collision fix
In reply to Hugh DickinsThe patch hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch used the value 0x40 for MAP_HUGETLB which is the same value used for various other flags on some architectures. This collision causes unexpected use of huge pages in the best case and mmap to fail with ENOMEM or ENOSYS in the worst. This patch changes the value for MAP_HUGETLB to a value that is not currently used on any arch.
This patch should be considered a fix to hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch.
--- include/asm-generic/mman-common.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
include/asm-generic/mman-common.h +1 -1
@@ -19,7 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ -#define MAP_HUGETLB 0x40/* create a huge page mapping */ +#define MAP_HUGETLB 0x080000/* create a huge page mapping */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH 1/1 Fix for hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch in -mm
Resending because this seems to have fallen between the cracks.
The patch hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch used the value 0x40 for MAP_HUGETLB which is the same value used for various other flags on some architectures. This collision causes unexpected use of huge pages in the best case and mmap to fail with ENOMEM or ENOSYS in the worst. This patch changes the value for MAP_HUGETLB to a value that is not currently used on any arch.
This patch should be considered a fix to hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch.
--- include/asm-generic/mman-common.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
include/asm-generic/mman-common.h +1 -1
@@ -19,7 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ -#define MAP_HUGETLB 0x40/* create a huge page mapping */ +#define MAP_HUGETLB 0x080000/* create a huge page mapping */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
Signature · 7 lines
1.6.3.2 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH] Fix for hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch in -mm
Quoted 1 line · Eric B Munson
Resending because this seems to have fallen between the cracks.
Yes, indeed. I think it isn't quite what Arnd was suggesting, but I agree with you that we might as well go for 0x080000 (so that even Alpha can be just a cut-and-paste job from asm-generic), and right now it's more important to finalize the number than what file it appears in.
Quoted 33 lines
The patch hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch used the value
0x40forMAP_HUGETLBwhich is the same value used for various other flags on some architectures. This collision causes unexpected use of huge pages in the best case and mmap to fail with ENOMEM or ENOSYS in the worst. This patch changes the value forMAP_HUGETLBto a value that is not currently used on any arch.This patch should be considered a fix to hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch.
Reported-by Hugh Dickins <[email protected]>Signed-off-by Eric B Munson <[email protected]>--- include/asm-generic/mman-common.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)include/asm-generic/mman-common.h +1
-1@@ -19,7 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ -#define MAP_HUGETLB 0x40/* create a huge page mapping */ +#define MAP_HUGETLB 0x080000/* create a huge page mapping */ #define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */ Signature · 1 line
1.6.3.2
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH] Fix for hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch in -mm
Quoted 9 lines · On Tue, 15 Sep 2009 21:53:12 +0100 (BST) Hugh Dickins wrote:
Quoted 1 line · Eric B Munson
Resending because this seems to have fallen between the cracks.
Yes, indeed. I think it isn't quite what Arnd was suggesting, but I agree with you that we might as well go for
0x080000(so that even Alpha can be just a cut-and-paste job from asm-generic), and right now it's more important to finalize the number than what file it appears in.Acked-by Hugh Dickins <[email protected]>
so what happened with this patch ??
Quoted 29 lines
Quoted 29 lines
The patch hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch used the value
0x40forMAP_HUGETLBwhich is the same value used for various other flags on some architectures. This collision causes unexpected use of huge pages in the best case and mmap to fail with ENOMEM or ENOSYS in the worst. This patch changes the value forMAP_HUGETLBto a value that is not currently used on any arch.This patch should be considered a fix to hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch.
Reported-by Hugh Dickins <[email protected]>Signed-off-by Eric B Munson <[email protected]>--- include/asm-generic/mman-common.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)include/asm-generic/mman-common.h +1
-1@@ -19,7 +19,7 @@#define MAP_TYPE 0x0f/* Mask for type of mapping */ #define MAP_FIXED 0x10/* Interpret addr exactly */ #define MAP_ANONYMOUS 0x20/* don't use a file */ -#define MAP_HUGETLB 0x40/* create a huge page mapping */ +#define MAP_HUGETLB 0x080000/* create a huge page mapping */
Quoted 3 lines
include/asm-generic/mman-common.h line 25
Quoted 3 lines
include/asm-generic/mman-common.h +0
-0#define MS_ASYNC 1/* sync memory asynchronously */ #define MS_INVALIDATE 2/* invalidate the caches */
---
~Randy
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Arnd Bergmann
maintainerRe: [PATCH] Fix for hugetlb-add-map_hugetlb-for-mmaping-pseudo-anonymous-huge-page-regions.patch in -mm
Quoted 5 lines · Randy Dunlap
Quoted 3 lines · On Tue, 15 Sep 2009 21:53:12 +0100 (BST) Hugh Dickins wrote:
Quoted 1 line · Eric B Munson
Resending because this seems to have fallen between the cracks.
Quoted 8 lines
Quoted 6 lines
Yes, indeed. I think it isn't quite what Arnd was suggesting, but I agree with you that we might as well go for
0x080000(so that even Alpha can be just a cut-and-paste job from asm-generic), and right now it's more important to finalize the number than what file it appears in.Acked-by Hugh Dickins <[email protected]>so what happened with this patch ??
In a later revision, we agreed to put the definition into asm-generic/mman.h, where it was merged in 2.6.32.
Arnd
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 1/3] hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on the vfs internal mount
In reply to Eric B MunsonQuoted 6 lines · Eric B Munson
There are two means of creating mappings backed by huge pages:
1.
mmap()a file created on hugetlbfs
2. Use shm which creates a file on an internal mount which essentially maps itMAP_SHARED
Quoted 6 lines
The internal mount is only used for shared mappings but there is very little that stops it being used for private mappings. This patch extends
hugetlbfs_file_setup()to deal with the creation of files that will be mappedMAP_PRIVATEon the internal hugetlbfs mount. This extended API is used in a subsequent patch to implement theMAP_HUGETLBmmap()flag.
Hi Eric,
I ran these patches through a series of small tests and I have just one concern with the changes made to can_do_hugetlb_shm(). If that returns false because of MAP_HUGETLB, we then proceed to call user_shm_lock(). I think your intention might have been something like the following patch on top of yours?
For what it's worth, once this was applied, I didn't spot any other problems, run-time or otherwise.
=====
hugetlbfs: Do not call user_shm_lock() for MAP_HUGETLB
The patch hugetlbfs-allow-the-creation-of-files-suitable-for-map_private-on-the-vfs-internal-mount.patch alters can_do_hugetlb_shm() to check if a file is being created for shared memory or mmap(). If this returns false, we then unconditionally call user_shm_lock() triggering a warning. This block should never be entered
for MAP_HUGETLB. This patch partially reverts the problem and fixes the check.
This patch should be considered a fix to hugetlbfs-allow-the-creation-of-files-suitable-for-map_private-on-the-vfs-internal-mount.patch.
---
fs/hugetlbfs/inode.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)
fs/hugetlbfs/inode.c +3 -9
@@ -910,15 +910,9 @@ static struct file_system_type hugetlbfs_fs_type = {static struct vfsmount *hugetlbfs_vfsmount; -static int can_do_hugetlb_shm (int creat_flags) +static int can_do_hugetlb_shm (void ) { -if (creat_flags !=HUGETLB_SHMFS_INODE ) -return 0 ; -if (capable (CAP_IPC_LOCK )) -return 1 ; -if (in_group_p (sysctl_hugetlb_shm_group)) -return 1 ; -return 0 ; +return capable (CAP_IPC_LOCK ) ||in_group_p (sysctl_hugetlb_shm_group); } struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag,
@@ -934,7 +928,7 @@ struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag,if (!hugetlbfs_vfsmount)return ERR_PTR (-ENOENT ); -if (!can_do_hugetlb_shm (creat_flags)) { +if (creat_flags ==HUGETLB_SHMFS_INODE && !can_do_hugetlb_shm ()) { *user =current_user ();if (user_shm_lock (size, *user)) { WARN_ONCE(1,
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH 1/3] hugetlbfs: Allow the creation of files suitable for MAP_PRIVATE on the vfs internal mount
Quoted 16 lines · Mel Gorman
Quoted 6 lines · Eric B Munson
There are two means of creating mappings backed by huge pages:
1.
mmap()a file created on hugetlbfs
2. Use shm which creates a file on an internal mount which essentially maps itMAP_SHAREDQuoted 6 lines
The internal mount is only used for shared mappings but there is very little that stops it being used for private mappings. This patch extends
hugetlbfs_file_setup()to deal with the creation of files that will be mappedMAP_PRIVATEon the internal hugetlbfs mount. This extended API is used in a subsequent patch to implement theMAP_HUGETLBmmap()flag.Hi Eric,
Quoted 8 lines
I ran these patches through a series of small tests and I have just one concern with the changes made to
can_do_hugetlb_shm(). If that returns false because ofMAP_HUGETLB, we then proceed to calluser_shm_lock(). I think your intention might have been something like the following patch on top of yours?For what it's worth, once this was applied, I didn't spot any other problems, run-time or otherwise.
I am seeing the same thing, terminal says segfault with no memory, dmesg complains about SHM. Your patch fixes the issue. Thanks.
Quoted 51 lines
=====
hugetlbfs: Do not calluser_shm_lock()forMAP_HUGETLBThe patch hugetlbfs-allow-the-creation-of-files-suitable-for-map_private-on-the-vfs-internal-mount.patch alters
can_do_hugetlb_shm()to check if a file is being created for shared memory ormmap(). If this returns false, we then unconditionally calluser_shm_lock()triggering a warning. This block should never be entered
forMAP_HUGETLB. This patch partially reverts the problem and fixes the check.This patch should be considered a fix to hugetlbfs-allow-the-creation-of-files-suitable-for-map_private-on-the-vfs-internal-mount.patch.
Signed-off-by Mel Gorman <[email protected]>---
fs/hugetlbfs/inode.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-)fs/hugetlbfs/inode.c +3
-9@@ -910,15 +910,9 @@ static struct file_system_type hugetlbfs_fs_type = {static struct vfsmount *hugetlbfs_vfsmount; -static int can_do_hugetlb_shm (int creat_flags) +static int can_do_hugetlb_shm (void ) { -if (creat_flags !=HUGETLB_SHMFS_INODE ) -return 0 ; -if (capable (CAP_IPC_LOCK )) -return 1 ; -if (in_group_p (sysctl_hugetlb_shm_group)) -return 1 ; -return 0 ; +return capable (CAP_IPC_LOCK ) ||in_group_p (sysctl_hugetlb_shm_group); } struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag,@@ -934,7 +928,7 @@ struct file *hugetlb_file_setup(const char *name, size_t size, int acctflag,if (!hugetlbfs_vfsmount)return ERR_PTR (-ENOENT ); -if (!can_do_hugetlb_shm (creat_flags)) { +if (creat_flags ==HUGETLB_SHMFS_INODE && !can_do_hugetlb_shm ()) { *user =current_user ();if (user_shm_lock (size, *user)) { WARN_ONCE(1,
Signature · 3 lines
Eric B Munson IBM Linux Technology Center [email protected]