Skip to contentSkip to conversation

sysctl_{tcp,udp,sctp}_mem overflow on 16TB system.

DISCUSSION Robin Holt 3 participants 5 messages Oct 1, 2010 – Oct 3, 2010

Subsystems (3)
lore
Share & tools
lore

Robin Holt

author
#0 lore raw

On a 16TB system, we noticed that sysctl_tcp_mem[2] and sysctl_udp_mem[2] were negative. Code review indicates that the same should occur with sysctl_sctp_mem[2].

There are a couple ways we could address this. The one which appears most reasonable would be to change the struct proto defintion for sysctl_mem from an int to a long and handle all the associated fallout.

An alternative is to limit the calculation to 1/2 INT_MAX. The downside being that the administrator could not tune the system to use more than INT_MAX memory when much more is available.

Is there a compelling reason to not change the structure's definition over to longs instead of ints and deal with the fallout from that change?

Thanks,
Robin Holt

#1 lore raw

Hello Robin,

Quoted 9 lines · Robin Holt

On a 16TB system, we noticed that sysctl_tcp_mem[2] and sysctl_udp_mem[2] were negative. Code review indicates that the same should occur with sysctl_sctp_mem[2].

There are a couple ways we could address this. The one which appears most reasonable would be to change the struct proto defintion for sysctl_mem from an int to a long and handle all the associated fallout.

Quoted 6 lines

An alternative is to limit the calculation to 1/2 INT_MAX. The downside being that the administrator could not tune the system to use more than INT_MAX memory when much more is available.

Is there a compelling reason to not change the structure's definition over to longs instead of ints and deal with the fallout from that change?

Could we not see it differently ? => is there any reason someone would want to assign more than 8 TB of RAM to the network buffers in the near future ? Even at 100 Gbps, that's still 10 minutes of traffic stuck in buffers. Probably that the day we need that large buffers, Linux won't support 32-bit systems anymore and all such limits will have switched to 64-bit.

So probably that limiting the value to INT_MAX/2 sounds reasonable ?

Regards,
Willy

author
#3 lore raw
Quoted 1 line · Maciej Å»enczykowski

Isn't INT_MAX/2 just 1GB, which is only ~0.9 seconds at 10 Gbps?

Units matter. 1GB pages. We can limit to 2GB pages or 8TB.

Robin

#4 lore raw
In reply to Maciej Żenczykowski
Quoted 1 line · Maciej ??enczykowski

Isn't INT_MAX/2 just 1GB, which is only ~0.9 seconds at 10 Gbps?

no, the size is in pages (and is often wrong when people change it from the default value).

Regards,
Willy