Sys V shared memory limited to 8TiB.
PATCH 1/1 Sys V shared memory limited to 8TiB.
Trying to run an application which was trying to put data into half of memory using shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.
In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.
ipc/shm.c:
458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params) 459 {
...
465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
...
474 if (ns->shm_tot + numpages > ns->shm_ctlall)
475 return -ENOSPC;
To: Andrew Morton <[email protected]>
Cc: Stable Kernel Maintainers <[email protected]>
Cc: lkml <[email protected]>
---
include/linux/ipc_namespace.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/ipc_namespace.h +1 -1
@@ -43,8 +43,8 @@ struct ipc_namespace {size_t shm_ctlmax;size_t shm_ctlall; +unsigned long shm_tot;int shm_ctlmni; -int shm_tot;/* * Defines whether IPC_RMID is forced for _all_ shm segments regardless * of shmctl()
Signature · 1 line
1.8.1.2
Quoted 6 lines · On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <[email protected]> wrote:
Trying to run an application which was trying to put data into half of memory using
shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.In the
newseg()function,ns->shm_totwhich, at 8TiB isINT_MAX.
You have way too much memory.
Quoted 14 lines
ipc/shm.c:
458 static intnewseg(struct ipc_namespace *ns, struct ipc_params *params)459 {
...
465 int numpages = (size +PAGE_SIZE-1) >>PAGE_SHIFT;
...
474 if (ns->shm_tot+ numpages >ns->shm_ctlall)
475 return-ENOSPC;...
include/linux/ipc_namespace.h struct ipc_namespace { · line 50
Quoted 9 lines
include/linux/ipc_namespace.h +1
-1@@ -43,8 +43,8 @@ struct ipc_namespace {size_t shm_ctlmax;size_t shm_ctlall; +unsigned long shm_tot;int shm_ctlmni; -int shm_tot;/* * Defines whether IPC_RMID is forced for _all_ shm segments regardless * of shmctl()
I reviewed everything for fallout from this and don't see any obvious issues.
I do wonder about the appropriateness of the unsigned long type. Most (but by no means all) code in this area uses size_t, and the above-quoted ns->shm_ctlall is size_t.
And the above-quoted num_pages is int'. Both the size and signedness of int' make no sense - what happens if the incoming ipc_params.u.size is >2G?
So I'll add this, and ask whether ipc_namespace.shm_tot should be size_t?
ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +1 -1
@@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *size_t size = params->u.size;int error;struct shmid_kernel *shp; -int numpages = (size +PAGE_SIZE -1 ) >>PAGE_SHIFT ; +size_t numpages = (size +PAGE_SIZE -1 ) >>PAGE_SHIFT ;struct file * file;char name[13 ];int id;
_
Quoted 38 lines · Andrew Morton
Quoted 6 lines · On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <[email protected]> wrote:
Trying to run an application which was trying to put data into half of memory using
shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.In the
newseg()function,ns->shm_totwhich, at 8TiB isINT_MAX.You have way too much memory.
Quoted 14 lines
ipc/shm.c:
458 static intnewseg(struct ipc_namespace *ns, struct ipc_params *params)459 {
...
465 int numpages = (size +PAGE_SIZE-1) >>PAGE_SHIFT;
...
474 if (ns->shm_tot+ numpages >ns->shm_ctlall)
475 return-ENOSPC;...
include/linux/ipc_namespace.h struct ipc_namespace { · line 50
Quoted 9 lines
include/linux/ipc_namespace.h +1
-1@@ -43,8 +43,8 @@ struct ipc_namespace {size_t shm_ctlmax;size_t shm_ctlall; +unsigned long shm_tot;int shm_ctlmni; -int shm_tot;/* * Defines whether IPC_RMID is forced for _all_ shm segments regardless * of shmctl() I reviewed everything for fallout from this and don't see any obvious issues.
Quoted 3 lines
I do wonder about the appropriateness of the unsigned long type. Most (but by no means all) code in this area uses
size_t, and the above-quotedns->shm_ctlallissize_t.
The only reason I went with unsigned long instead of size_t was most places in the kernel track stuff I recalled that was tracking stuff in pages used unsigned longs. Also, I found shm_tot field in shm_info structure was an unsigned long so this felt like a natural fit. I would happily changed to size_t. Whatever you feel is right.
Quoted 9 lines
And the above-quoted
num_pagesisint'. Both the size and signedness ofint' make no sense - what happens if the incoming ipc_params.u.size is >2G?So I'll add this, and ask whether ipc_namespace.shm_tot should be
size_t?
ipc/shm.c static int newseg(struct ipc_namespace * · line 465
Quoted 5 lines
ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +1
-1@@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *size_t size = params->u.size;int error;struct shmid_kernel *shp; -int numpages = (size +PAGE_SIZE -1 ) >>PAGE_SHIFT ; +size_t numpages = (size +PAGE_SIZE -1 ) >>PAGE_SHIFT ;
I was holding off from that change only because I was asking for this to go to stable and this doubles the size of the patch. ;)
Quoted 4 lines
ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +0
-0struct file * file;char name[13 ];int id;_
Thank you,
Robin
Quoted 43 lines · On Wed, 10 Apr 2013 21:42:23 -0500 Robin Holt <[email protected]> wrote:
Quoted 38 lines · Andrew Morton
Quoted 6 lines · On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <[email protected]> wrote:
Trying to run an application which was trying to put data into half of memory using
shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.In the
newseg()function,ns->shm_totwhich, at 8TiB isINT_MAX.You have way too much memory.
Quoted 14 lines
ipc/shm.c:
458 static intnewseg(struct ipc_namespace *ns, struct ipc_params *params)459 {
...
465 int numpages = (size +PAGE_SIZE-1) >>PAGE_SHIFT;
...
474 if (ns->shm_tot+ numpages >ns->shm_ctlall)
475 return-ENOSPC;...
include/linux/ipc_namespace.h struct ipc_namespace { · line 50
Quoted 9 lines
include/linux/ipc_namespace.h +1
-1@@ -43,8 +43,8 @@ struct ipc_namespace {size_t shm_ctlmax;size_t shm_ctlall; +unsigned long shm_tot;int shm_ctlmni; -int shm_tot;/* * Defines whether IPC_RMID is forced for _all_ shm segments regardless * of shmctl() I reviewed everything for fallout from this and don't see any obvious issues.
Quoted 3 lines
I do wonder about the appropriateness of the unsigned long type. Most (but by no means all) code in this area uses
size_t, and the above-quotedns->shm_ctlallissize_t.
Quoted 5 lines
The only reason I went with unsigned long instead of
size_twas most places in the kernel track stuff I recalled that was tracking stuff in pages used unsigned longs. Also, I foundshm_totfield inshm_infostructure was an unsigned long so this felt like a natural fit. I would happily changed tosize_t. Whatever you feel is right.
I have no really strong feelings, but let's at least put some thought into it. I do prefer ulong and find size_t to be a PITA. I guess it doesn't matter much.
Quoted 11 lines
ipc/shm.c static int newseg(struct ipc_namespace * · line 465
Quoted 8 lines
ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +1
-1@@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *size_t size = params->u.size;int error;struct shmid_kernel *shp; -int numpages = (size +PAGE_SIZE -1 ) >>PAGE_SHIFT ; +size_t numpages = (size +PAGE_SIZE -1 ) >>PAGE_SHIFT ;I was holding off from that change only because I was asking for this to go to stable and this doubles the size of the patch. ;)
It's a bug, isn't it? Is there anything else which prevents creation of segments which are >=8TB?
Quoted 67 lines · Andrew Morton
Quoted 43 lines · On Wed, 10 Apr 2013 21:42:23 -0500 Robin Holt <[email protected]> wrote:
Quoted 38 lines · Andrew Morton
Quoted 6 lines · On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <[email protected]> wrote:
Trying to run an application which was trying to put data into half of memory using
shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.In the
newseg()function,ns->shm_totwhich, at 8TiB isINT_MAX.You have way too much memory.
Quoted 14 lines
ipc/shm.c:
458 static intnewseg(struct ipc_namespace *ns, struct ipc_params *params)459 {
...
465 int numpages = (size +PAGE_SIZE-1) >>PAGE_SHIFT;
...
474 if (ns->shm_tot+ numpages >ns->shm_ctlall)
475 return-ENOSPC;...
include/linux/ipc_namespace.h struct ipc_namespace { · line 50
Quoted 9 lines
include/linux/ipc_namespace.h +1
-1@@ -43,8 +43,8 @@ struct ipc_namespace {size_t shm_ctlmax;size_t shm_ctlall; +unsigned long shm_tot;int shm_ctlmni; -int shm_tot;/* * Defines whether IPC_RMID is forced for _all_ shm segments regardless * of shmctl() I reviewed everything for fallout from this and don't see any obvious issues.
Quoted 3 lines
I do wonder about the appropriateness of the unsigned long type. Most (but by no means all) code in this area uses
size_t, and the above-quotedns->shm_ctlallissize_t.Quoted 5 lines
The only reason I went with unsigned long instead of
size_twas most places in the kernel track stuff I recalled that was tracking stuff in pages used unsigned longs. Also, I foundshm_totfield inshm_infostructure was an unsigned long so this felt like a natural fit. I would happily changed tosize_t. Whatever you feel is right.I have no really strong feelings, but let's at least put some thought into it. I do prefer ulong and find
size_tto be a PITA. I guess it doesn't matter much.Quoted 11 lines
ipc/shm.c static int newseg(struct ipc_namespace * · line 465
Quoted 8 lines
ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +1
-1@@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *size_t size = params->u.size;int error;struct shmid_kernel *shp; -int numpages = (size +PAGE_SIZE -1 ) >>PAGE_SHIFT ; +size_t numpages = (size +PAGE_SIZE -1 ) >>PAGE_SHIFT ;I was holding off from that change only because I was asking for this to go to stable and this doubles the size of the patch. ;)
Quoted 2 lines
It's a bug, isn't it? Is there anything else which prevents creation of segments which are >=8TB?
Definitely a bug. Have not tried to create a segment >= 8TiB. I can give that a try tomorrow morning when I have a machine again to test with.
Thanks,
Robin