Forking shell bombs

The problem can be attacked in two steps:

1. Stop new forks from being created
2. Kill the process causing the forks

The current ulimit implementation, afaik, can only control the processes which will be created from the current moment onwards. Ther processes which are already started will continue creating forks. New processes created by the fork wil have this limit.

Basically it does not ensure that first step is completely executed. So if your rate of killing is less than the processes being created and resources are exausted, your system hangs.

There was a RFC patch "[RFC][PATCH 2.5.70] Dynamically tunable
maxusers, maxuprc and max_pt_cnt" posted on 2003-06-06. It implements maxuprc (maxuprc: maximum number of processes per user) as a dynamic tunable parameter. It can be useful to overcome this problem. By setting maxuprc to a very low value, new creation of the process is stopped. Then root can kill ('cos this limit is not applicable to root) the erring processes.
There is no race against time as there is no chance of new process getting created once this value is reduced.

cheers ..
Arvind ...