Virtual Console Hotplug Leads to OOM

Colleagues,

I've identified the following critical issue where the hotplug subsystem can cause OOM-killer to run. IMO this is a critical issue within hotplug -- in the case below, only the virtual console subsystem is responsible.

The virtual consoles that are being spawned are responsible for 2 hotplug events for every console opened -- one on the opening of the console, and one on the closing of the console.

If you run the tty2_test which opens and closes ttys (attached to this email) you'll see THOUSANDS of hotplug events. After running for 1 minute on a 16p ia64 I had well over 4000 hotplug events.

If we're unlucky enough to fall below the low-water memory point at the same time tty2_test is spawning, the OOM-killer starts up. Memory is taken up by the hotplug events ... and we're off into the weeds.

The issue with the virtual console code is that it uses the class_simple library. This is a problem as the library pre-defines a global kset parent as class_obj (or at least AFAICT). This parent is used by ALL subsystems using the class_simple, not just the virtual console subsystem.

This parent has a pointer to allow hotplug slot operations for all it's children (including the virtual consoles). IMO zero-ing this pointer out is not a valid option as this would break hotplug op's for many other subystems.

** I did notice that the class struct has it's own hotplug value -- can/should this be used to override the parent's hotplug operations? **

P.

-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_ide95&alloc_id396&op=click
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

It helps to attach the test code :)

P.

#include <sys/types.h>
#include <sys/stat.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <time.h>
#include <pthread.h>

#define NTHREADS 300

void *thread_function();
int open_fail_num;
int open_success;

int
main(int argc, char *argv[])
{
int i, j;
pthread_t thread_id[NTHREADS];

for(;;)

    {
      for(i=0; i < NTHREADS; i++)
	{
	  pthread_create(&thread_id[i], NULL, &thread_function, NULL);
	}
      for(j=0; j < NTHREADS; j++)
	{
	  pthread_join(thread_id[j], NULL); 
	}
      printf("open failures: %i\n", open_fail_num);
      printf("open success: %i\n", open_success);
    }
}

void *thread_function()
{
int fd;
time_t t;

  //  fd = open("/dev/cua0", O_NONBLOCK);
  //  fd = open("/dev/cua0", O_RDWR);
  fd = open("/dev/tty9", O_RDWR);

// if (fd == -1)
if (fd < 0)

    {
      open_fail_num++;
      //      perror("open");
      //      exit(1);
    }
  else
    {
      open_success++;
    }
  /* just waste some random time */
  t = (time((time_t *)0) &31L) << 6;
  while (t-- > 0)
    (void)time((time_t *)0);

close(fd);
// _exit(0);
}

#

# make test command
#
NAME_SRC=tty2_test
CFLAGS= -O -g -Wall -I /usr/include/
LDFLAGS=-lpthread
CC=cc
GET=co
SRCS=$(NAME_SRC).c
OBJS=$(NAME_SRC).o
DEPEND= makedepend $(CFLAGS)
all:	$(NAME_SRC)

# To make an executable

$(NAME_SRC): $(OBJS)

	$(CC) $(LDFLAGS) -o $@ $(OBJS)

# what are the source dependencies
depend:	$(SRCS)
	$(DEPEND) $(SRCS)

# clean out the junk
clean:
	-rm $(NAME_SRC) *~ *.o

# DO NOT DELETE THIS LINE -- make depend depends on it. $(NAME_SRC).o: /usr/include/sys/types.h /usr/include/sys/stat.h \
/usr/include/fcntl.h /usr/include/pthread.h

Yes.

Guess you need a faster machine, my little x86-64 had way more than that :)

Yup.

If you look at the -mm tree, class_simple is now gone.

Very true.

Why would you want to override this? We need those hotplug events. All this means is that an admin better be careful who he lets have access to the tty group :)

thanks,

greg k-h

-------------------------------------------------------
This SF.net email is sponsored by Demarc:
A global provider of Threat Management Solutions.
Download our HomeAdmin security software for free today!
http://www.demarc.com/info/Sentarus/hamr30
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Greg,

Thanks for the reply.

(sheepish grin) I did put a few sleeps in there so I could actually read the output streaming out :)

No offense, but I'm not sure we should be so cavalier on this issue -- I'm pointing out that we have a potential fork bomb within the kernel.

Aside from that, I'm think that hotplug is more of a victim here than the cause. I'm also noticing that on this machine with 100,000 outstanding task_structs, approximately 99,000 of them are hotplug task_structs. Something is definately very wrong here. These should have been reaped. These tasks are not in the pid table but are left lying around on the task_struct list.

Will do. Was there some signifigant reason for removing class_simple? (Just curious ...)

Absolutely correct -- but on some machines the group can be uucp ... which could *potentially* cause a security hole. The tty2_test is (very) contrived but still ... it should be something we're aware of as it could cause problems on machines at critical stress points.

-------------------------------------------------------
This SF.net email is sponsored by Demarc:
A global provider of Threat Management Solutions.
Download our HomeAdmin security software for free today!
http://www.demarc.com/info/Sentarus/hamr30
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel