Skip to contentSkip to conversation

Sys V shared memory limited to 8TiB.

PATCH In review Robin Holt 2 participants 5 messages Apr 10, 2013 – Apr 11, 2013

lore
Share & tools
lore

Robin Holt

author
#0 lore raw

PATCH 1/1 Sys V shared memory limited to 8TiB.

Trying to run an application which was trying to put data into half of memory using shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.

In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.

ipc/shm.c:
458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params) 459 {
...
465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
...
474 if (ns->shm_tot + numpages > ns->shm_ctlall)
475 return -ENOSPC;

To: Andrew Morton <[email protected]>
Cc: Stable Kernel Maintainers <[email protected]>
Cc: lkml <[email protected]>

---

 include/linux/ipc_namespace.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/ipc_namespace.h +1 -1
index ae221a7..ca62eca 100644
@@ -43,8 +43,8 @@ struct ipc_namespace {
 
 	size_t		shm_ctlmax;
 	size_t		shm_ctlall;
+	unsigned long	shm_tot;
 	int		shm_ctlmni;
-	int		shm_tot;
 	/*
 	 * Defines whether IPC_RMID is forced for _all_ shm segments regardless
 	 * of shmctl()
Signature · 1 line
1.8.1.2
#1 lore raw
Quoted 6 lines · On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <[email protected]> wrote:

Trying to run an application which was trying to put data into half of memory using shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.

In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.

You have way too much memory.

Quoted 14 lines

ipc/shm.c:
458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params) 459 {
...
465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
...
474 if (ns->shm_tot + numpages > ns->shm_ctlall)
475 return -ENOSPC;

...

include/linux/ipc_namespace.h struct ipc_namespace { · line 50

Quoted 9 lines
include/linux/ipc_namespace.h +1 -1
@@ -43,8 +43,8 @@ struct ipc_namespace {
 
 	size_t		shm_ctlmax;
 	size_t		shm_ctlall;
+	unsigned long	shm_tot;
 	int		shm_ctlmni;
-	int		shm_tot;
 	/*
 	 * Defines whether IPC_RMID is forced for _all_ shm segments regardless
 	 * of shmctl()

I reviewed everything for fallout from this and don't see any obvious issues.

I do wonder about the appropriateness of the unsigned long type. Most (but by no means all) code in this area uses size_t, and the above-quoted ns->shm_ctlall is size_t.

And the above-quoted num_pages is int'. Both the size and signedness of int' make no sense - what happens if the incoming ipc_params.u.size is >2G?

So I'll add this, and ask whether ipc_namespace.shm_tot should be size_t?

ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +1 -1
@@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *
 	size_t size = params->u.size;
 	int error;
 	struct shmid_kernel *shp;
-	int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
+	size_t numpages = (size + PAGE_SIZE - 1) >> PAGE_SHIFT;
 	struct file * file;
 	char name[13];
 	int id;

_

author
#2 lore raw
Quoted 38 lines · Andrew Morton
Quoted 6 lines · On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <[email protected]> wrote:

Trying to run an application which was trying to put data into half of memory using shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.

In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.

You have way too much memory.

Quoted 14 lines

ipc/shm.c:
458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params) 459 {
...
465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
...
474 if (ns->shm_tot + numpages > ns->shm_ctlall)
475 return -ENOSPC;

...

include/linux/ipc_namespace.h struct ipc_namespace { · line 50

Quoted 9 lines
include/linux/ipc_namespace.h +1 -1
@@ -43,8 +43,8 @@ struct ipc_namespace {
 
 	size_t		shm_ctlmax;
 	size_t		shm_ctlall;
+	unsigned long	shm_tot;
 	int		shm_ctlmni;
-	int		shm_tot;
 	/*
 	 * Defines whether IPC_RMID is forced for _all_ shm segments regardless
 	 * of shmctl()

I reviewed everything for fallout from this and don't see any obvious issues.

Quoted 3 lines

I do wonder about the appropriateness of the unsigned long type. Most (but by no means all) code in this area uses size_t, and the above-quoted ns->shm_ctlall is size_t.

The only reason I went with unsigned long instead of size_t was most places in the kernel track stuff I recalled that was tracking stuff in pages used unsigned longs. Also, I found shm_tot field in shm_info structure was an unsigned long so this felt like a natural fit. I would happily changed to size_t. Whatever you feel is right.

Quoted 9 lines

And the above-quoted num_pages is int'. Both the size and signedness of int' make no sense - what happens if the incoming ipc_params.u.size is >2G?

So I'll add this, and ask whether ipc_namespace.shm_tot should be size_t?

ipc/shm.c static int newseg(struct ipc_namespace * · line 465

Quoted 5 lines
ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +1 -1
@@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *
 	size_t size = params->u.size;
 	int error;
 	struct shmid_kernel *shp;
-	int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
+	size_t numpages = (size + PAGE_SIZE - 1) >> PAGE_SHIFT;

I was holding off from that change only because I was asking for this to go to stable and this doubles the size of the patch. ;)

Quoted 4 lines
ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +0 -0
 	struct file * file;
 	char name[13];
 	int id;

_

Thank you,
Robin

#3 lore raw
Quoted 43 lines · On Wed, 10 Apr 2013 21:42:23 -0500 Robin Holt <[email protected]> wrote:
Quoted 38 lines · Andrew Morton
Quoted 6 lines · On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <[email protected]> wrote:

Trying to run an application which was trying to put data into half of memory using shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.

In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.

You have way too much memory.

Quoted 14 lines

ipc/shm.c:
458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params) 459 {
...
465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
...
474 if (ns->shm_tot + numpages > ns->shm_ctlall)
475 return -ENOSPC;

...

include/linux/ipc_namespace.h struct ipc_namespace { · line 50

Quoted 9 lines
include/linux/ipc_namespace.h +1 -1
@@ -43,8 +43,8 @@ struct ipc_namespace {
 
 	size_t		shm_ctlmax;
 	size_t		shm_ctlall;
+	unsigned long	shm_tot;
 	int		shm_ctlmni;
-	int		shm_tot;
 	/*
 	 * Defines whether IPC_RMID is forced for _all_ shm segments regardless
 	 * of shmctl()

I reviewed everything for fallout from this and don't see any obvious issues.

Quoted 3 lines

I do wonder about the appropriateness of the unsigned long type. Most (but by no means all) code in this area uses size_t, and the above-quoted ns->shm_ctlall is size_t.

Quoted 5 lines

The only reason I went with unsigned long instead of size_t was most places in the kernel track stuff I recalled that was tracking stuff in pages used unsigned longs. Also, I found shm_tot field in shm_info structure was an unsigned long so this felt like a natural fit. I would happily changed to size_t. Whatever you feel is right.

I have no really strong feelings, but let's at least put some thought into it. I do prefer ulong and find size_t to be a PITA. I guess it doesn't matter much.

Quoted 11 lines

ipc/shm.c static int newseg(struct ipc_namespace * · line 465

Quoted 8 lines
ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +1 -1
@@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *
 	size_t size = params->u.size;
 	int error;
 	struct shmid_kernel *shp;
-	int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
+	size_t numpages = (size + PAGE_SIZE - 1) >> PAGE_SHIFT;

I was holding off from that change only because I was asking for this to go to stable and this doubles the size of the patch. ;)

It's a bug, isn't it? Is there anything else which prevents creation of segments which are >=8TB?

author
#4 lore raw
Quoted 67 lines · Andrew Morton
Quoted 43 lines · On Wed, 10 Apr 2013 21:42:23 -0500 Robin Holt <[email protected]> wrote:
Quoted 38 lines · Andrew Morton
Quoted 6 lines · On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <[email protected]> wrote:

Trying to run an application which was trying to put data into half of memory using shmget(), we found that having a shmall value below 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting kernel.shmall greater that 8EiB-8TiB would make the job work.

In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.

You have way too much memory.

Quoted 14 lines

ipc/shm.c:
458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params) 459 {
...
465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
...
474 if (ns->shm_tot + numpages > ns->shm_ctlall)
475 return -ENOSPC;

...

include/linux/ipc_namespace.h struct ipc_namespace { · line 50

Quoted 9 lines
include/linux/ipc_namespace.h +1 -1
@@ -43,8 +43,8 @@ struct ipc_namespace {
 
 	size_t		shm_ctlmax;
 	size_t		shm_ctlall;
+	unsigned long	shm_tot;
 	int		shm_ctlmni;
-	int		shm_tot;
 	/*
 	 * Defines whether IPC_RMID is forced for _all_ shm segments regardless
 	 * of shmctl()

I reviewed everything for fallout from this and don't see any obvious issues.

Quoted 3 lines

I do wonder about the appropriateness of the unsigned long type. Most (but by no means all) code in this area uses size_t, and the above-quoted ns->shm_ctlall is size_t.

Quoted 5 lines

The only reason I went with unsigned long instead of size_t was most places in the kernel track stuff I recalled that was tracking stuff in pages used unsigned longs. Also, I found shm_tot field in shm_info structure was an unsigned long so this felt like a natural fit. I would happily changed to size_t. Whatever you feel is right.

I have no really strong feelings, but let's at least put some thought into it. I do prefer ulong and find size_t to be a PITA. I guess it doesn't matter much.

Quoted 11 lines

ipc/shm.c static int newseg(struct ipc_namespace * · line 465

Quoted 8 lines
ipc/shm.c renamed from ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix +1 -1
@@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *
 	size_t size = params->u.size;
 	int error;
 	struct shmid_kernel *shp;
-	int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
+	size_t numpages = (size + PAGE_SIZE - 1) >> PAGE_SHIFT;

I was holding off from that change only because I was asking for this to go to stable and this doubles the size of the patch. ;)

Quoted 2 lines

It's a bug, isn't it? Is there anything else which prevents creation of segments which are >=8TB?

Definitely a bug. Have not tried to create a segment >= 8TiB. I can give that a try tomorrow morning when I have a machine again to test with.

Thanks,
Robin