processes with shared vm

The basic idea is a good one (I have written a similar thing in the past ;) Your implementation is O(n^2) however in ps, which is not acceptable. Much better is it to add a new field to mm_struct that gets initialised on first creation with the pid, and adding a place holder in pid hash if that process goes away and the mm_struct is still there to avoid pid reuse (or alternatively link task_structs to mms and always use the pid of the first entry)

-Andi

Is there any reason your patch was not accepted? Perhaps for 2.5?

This is something (along with userspace changes to take advtantage of it) that I think is really needed -- no more bogus ps/top reports.

I liked Terje's idea, but obviously the scalability needs to be improved (I didn't even notice it, sadly). I would really want to see this at some point.

Robert M. Love
rml at ufl.edu
rml at tech9.net

Den 17 Aug 2001 10:21:35 +0200, skrev Andi Kleen:

Thought of all that, yes ps will have O(n^2) BUT ONLY FOR CLONED PROCS. How many cloned procs do you usually have????

Even if I agree that there should be a linked list of all the cloned procs, it means major changes to the data structs in the kernel.

With the number of threaded programs out there, this is "good enough".

TJ

_________________________________________________________________________

Terje Eggestad                  [email protected]
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 70 Bogerud                      +47 975 31 574  (MOBILE)
N-0621 Oslo                     fax:    +47 22 62 89 51
NORWAY            
_________________________________________________________________________

There is a simpler way to do this. All tasks belong to a thread group, and while thread groups are connected via a different clone flag (CLONE_THREAD), in practice CLONE_THREAD and CLONE_VM are generally used together. It would be trivial to add TGID to the information in /proc, then assume all tasks with the same TGID have the same VM as well. It would be just one more line in the /proc output and not require any additional overhead.

Dave McCracken

======================================================================
Dave McCracken IBM Linux Base Kernel Team 1-512-838-3059
[email protected] T/L 678-3059

Even with a tgid you would need some way to avoid its counter wrapping and getting reused. I think reusing the pid and pid hash for that is cheaper. Also gtop should display correct results even with the programs that don't use CLONE_THREAD.

-Andi

While in theory the pid that is used for tgid should never die while the thread group exists, this case needs to be handled for thread groups in general. The number shouldn't be re-used for a pid as long as it's in use as a tgid.

Are there any programs that use CLONE_VM and not CLONE_THREAD?

Actually I think we should make tgid visible in /proc in general because it's a useful thing to know, whether it's the right mechanism for gtop or not. I'll work up a patch.

Dave McCracken

======================================================================
Dave McCracken IBM Linux Base Kernel Team 1-512-838-3059
[email protected] T/L 678-3059

Be careful, I tested by using _clone() dircetly and let a proc A create two clones B and C. I then let A die, and B &C's ppid was now 1. Attaching this code, you may find it useful for your tgid patch. (this code is pretty unclean, ran into the problem that if A cloned B and B cloned C, and now B died B remained a zombie and I didn't seem to be able to catch SIGCHLD in A. (wanted to se C's ppid go to 1), probably something stupid, just didn't bother to pursue it....)

THeory is correct if every one is using pthreads... (exactly how many thread libs are ut there.....??)

attached program ;-)
Maybe not, but *the kernel allows it*!

Looking forward to it!

TJ

_________________________________________________________________________

Terje Eggestad                  [email protected]
Scali Scalable Linux Systems    http://www.scali.com

Olaf Helsets Vei 6              tel:    +47 22 62 89 61 (OFFICE)
P.O.Box 70 Bogerud                      +47 975 31 574  (MOBILE)
N-0621 Oslo                     fax:    +47 22 62 89 51
NORWAY
_________________________________________________________________________

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <sys/types.h>
#include <signal.h>
#include <sched.h>

pid_t pid0, pid1, pid2;
char * stack1, * stack2;

#define STACKSIZE 1024*8*4

struct sigcall{
int sig;
pid_t pid;
} siglist[1024];
int nsig = 0;

printfsiglist()
{
int i;
for (i = 0; i< nsig; i++)

    printf("%d:: %d: %d %d\n", getpid(), i, siglist[nsig].pid, siglist[nsig].sig);
};

void sighand(int sig)
{
siglist[nsig].pid = getpid();
siglist[nsig].sig = sig;
nsig++;
}

int clone2()
{

  pid2 = getpid();
  printf("clone2 has pid %d\n", pid2);

sleep(100);
printfsiglist();
};

int clone1()
{
char * stack;

  pid1 = getpid();
  printf("clone1 has pid %d\n", pid1);

sleep(10);
printfsiglist();
return;
exit(0);
};

main()
{
char * stack;
int x, rc;
pid0 = getpid();
printf("clone0 has pid %d\n", pid0);

  signal(SIGCHLD, sighand);
  signal(SIGALRM, sighand);
  stack1 = malloc(STACKSIZE);
  stack2 = malloc(STACKSIZE);
  stack1 += STACKSIZE;
  stack2 += STACKSIZE;
  pid1 = __clone(clone1, stack1, CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND, NULL);
  pid2 = __clone(clone2, stack2, CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND, NULL);

  sleep(1);
  printf("master: clone0 has pid %d\n", pid0);
  printf("master: clone1 has pid %d\n", pid1);
  printf("master: clone2 has pid %d\n", pid2);

  exit(2);

};

Yes, I have written an emulation layer for an embedded OS which does its own preemptive scheduling using signals. It uses CLONE_VM but not CLONE_THREAD or CLONE_SIGHAND. Currently it is very painful to debug with gdb :-(, although the multithreaded core dumps in -ac are a godsend.

miket