Skip to contentSkip to conversation

Increase size of user stack

PATCH David Mosberger 4 participants 7 messages May 5, 2003 – May 12, 2003

lore
Share & tools
lore
#0 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Martin Hicks

Martin> The current user stack size limit of 2GB is not big enough Martin> for large SMP or SSI clusters. The following patch Martin> increases this size from 2GB to 512GB. The new size is Martin> negotiable, but right now SGI has programs that require over Martin> 100GB of stack size on a 64 processor machine. Setting the Martin> user stack size to 512GB seems like a reasonable thing to do Martin> for future growth.

Martin> Comments?

Why not make it configurable via setrlimit() and be done with it once and for all?

	--david
#1 lore raw
Quoted 10 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Martin Hicks

Martin> The current user stack size limit of 2GB is not big enough Martin> for large SMP or SSI clusters. The following patch Martin> increases this size from 2GB to 512GB. The new size is Martin> negotiable, but right now SGI has programs that require over Martin> 100GB of stack size on a 64 processor machine. Setting the Martin> user stack size to 512GB seems like a reasonable thing to do Martin> for future growth.

Quoted 2 lines

Why not make it configurable via setrlimit() and be done with it once and for all?

(sorry for taking so long to respond about this)

The user stack size is configurable through setrlimit(). The default size of the user stack is 8MB and the max is ~0UL. The problem is the addreses that are chosen for the stack (which grows down) and the register backing store (which grows up) are only 2GB apart. My patch simply makes this space 512GB so that large stack are possible, without the stack smashing into the RBS.

mh

Signature · 1 line
Wild Open Source Inc.                  [email protected]
#2 lore raw
In reply to David Mosberger
Quoted 12 lines · Martin Hicks
Quoted 10 lines · David Mosberger

Martin> The current user stack size limit of 2GB is not big enough Martin> for large SMP or SSI clusters. The following patch Martin> increases this size from 2GB to 512GB. The new size is Martin> negotiable, but right now SGI has programs that require over Martin> 100GB of stack size on a 64 processor machine. Setting the Martin> user stack size to 512GB seems like a reasonable thing to do Martin> for future growth.

Why not make it configurable via setrlimit() and be done with it once and for all?

Quoted 8 lines

(sorry for taking so long to respond about this)

The user stack size is configurable through setrlimit(). The default size of the user stack is 8MB and the max is ~0UL. The problem is the addreses that are chosen for the stack (which grows down) and the register backing store (which grows up) are only 2GB apart. My patch simply makes this space 512GB so that large stack are possible, without the stack smashing into the RBS.

Yes, but why not choose the addresses based on the current setting of current->rlim[RLIMIT_STACK].rlim_max? We could default that setting to 2GB (ie the same as what people are using now) and then set the RBS and the user stack min(1TB, current->rlim[RLIMIT_STACK].rlim_max) apart. Are there any disadvantages to doing this?

Signature · 3 lines
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk
#3 lore raw
In reply to David Mosberger
Quoted 11 lines · Matthew Wilcox
Quoted 9 lines · Martin Hicks
Quoted 1 line

(sorry for taking so long to respond about this)

The user stack size is configurable through setrlimit(). The default size of the user stack is 8MB and the max is ~0UL. The problem is the addreses that are chosen for the stack (which grows down) and the register backing store (which grows up) are only 2GB apart. My patch simply makes this space 512GB so that large stack are possible, without the stack smashing into the RBS.

Quoted 5 lines

Yes, but why not choose the addresses based on the current setting of current->rlim[RLIMIT_STACK].rlim_max? We could default that setting to 2GB (ie the same as what people are using now) and then set the RBS and the user stack min(1TB, current->rlim[RLIMIT_STACK].rlim_max) apart. Are there any disadvantages to doing this?

Maybe I'm missing something, but what happens if a process is created using the default 2GB between the RBS and the stack and sometime later the process does a setrlimit to 100GB of stack. How do the stack and RBS get moved around to make enough room between the two start addresses?

mh

Signature · 1 line
Wild Open Source Inc.                  [email protected]
#4 lore raw
In reply to David Mosberger
Quoted 5 lines · Martin Hicks

Maybe I'm missing something, but what happens if a process is created using the default 2GB between the RBS and the stack and sometime later the process does a setrlimit to 100GB of stack. How do the stack and RBS get moved around to make enough room between the two start addresses?

They don't. However, the new maximum would take effect for any newly created children.

Signature · 3 lines
"It's not Hollywood.  War is real, war is primarily not about defeat or
victory, it is about death.  I've seen thousands and thousands of dead bodies.
Do you think I want to have an academic debate on this subject?" -- Robert Fisk
#5 lore raw
In reply to David Mosberger

Matthew Wilcox <[email protected]> writes:

|> On Mon, May 12, 2003 at 10:59:38AM -0400, Martin Hicks wrote:
|> > Maybe I'm missing something, but what happens if a process is created
|> > using the default 2GB between the RBS and the stack and sometime
|> > later the process does a setrlimit to 100GB of stack.  How do the 
|> > stack and RBS get moved around to make enough room between the two
|> > start addresses?
|> 
|> They don't.  However, the new maximum would take effect for any newly
|> created children.

But setrlimit takes effect immediately, not only for newly created children.

Andreas.

Signature · 4 lines
Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#6 lore raw
In reply to David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Martin Hicks

Martin> The user stack size is configurable through setrlimit(). Martin> The default size of the user stack is 8MB and the max is Martin> ~0UL. The problem is the addreses that are chosen for the Martin> stack (which grows down) and the register backing store Martin> (which grows up) are only 2GB apart. My patch simply makes Martin> this space 512GB so that large stack are possible, without Martin> the stack smashing into the RBS.

OK, time for a pop-quiz:

Q: You are faced with a timebomb that's set to go of in 1 minute.
What do you do?

(a) Tweak the timer so it goes off in ~4 1/4 hours instead and get the hell out of town.

(b) Remove the timing device so it doesn't go off at all.

If you're answer is (a), I'm afraid you stand an excellent chance of getting voted off the island.

Clear enough? ;-)

	--david