udev hangs under high loads
One of my coworkers (hi Nish) is trying to get 10000 disk support tested and working properly with linux and ran into a nasty udev issue yesterday. It seems that under a very high load, with a whole bunch of hotplug events happening (for when the disks get added to the system), udev hangs.
It hangs in the "grabbing the database lock" portion of the code (don't have actual logs of where it hangs, will try to get that next week.) But the interesting thing is we almost handle everything properly. udev creates the node, tries to write to the database. Then the timer expires and we report this. After the timer expires, udev is done for. It just sits and spins, doing a nanosleep constantly. Have 500 of those instances all running at once, all at a nice level of -10 is a sure way to bring a box (even a relatively big one) down hard.
So, while I'll agree finding the root tdb locking bug is a good idea, I wanted to point out that perhaps we should just exit udev if our timeout expires, instead of still waiting around. Or do you have a better solution?
Try testing this out on your own using the scsi_debug module and adding a few hundred disks. It also helps if you have scsi generic support enabled, as that creates 2 udev events for every disk created.
thanks,
greg k-h
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 15 lines · Greg KH
One of my coworkers (hi Nish) is trying to get 10000 disk support tested and working properly with linux and ran into a nasty udev issue yesterday. It seems that under a very high load, with a whole bunch of hotplug events happening (for when the disks get added to the system), udev hangs.
It hangs in the "grabbing the database lock" portion of the code (don't have actual logs of where it hangs, will try to get that next week.) But the interesting thing is we almost handle everything properly. udev creates the node, tries to write to the database. Then the timer expires and we report this. After the timer expires, udev is done for. It just sits and spins, doing a nanosleep constantly. Have 500 of those instances all running at once, all at a nice level of -10 is a sure way to bring a box (even a relatively big one) down hard.
Quoted 4 lines
So, while I'll agree finding the root tdb locking bug is a good idea, I wanted to point out that perhaps we should just exit udev if our timeout expires, instead of still waiting around. Or do you have a better solution?
Maybe the time the udev process locks the db is too long for that setup and the serialization of the concurrent locks will take longer than the timout of the process?
Here is a patch that tries to limit the access time to the db. The current udev opens the db, reads all the rules, writes to the db, executes the scripts and then closes the db. With this patch we open the db after the rules and close it directly after writing to it.
A rate limit in udevd may help here too, to keep that under control.
Quoted 3 lines
Try testing this out on your own using the
scsi_debugmodule and adding a few hundred disks. It also helps if you have scsi generic support enabled, as that creates 2 udev events for every disk created.
I expect that this is completely different on a SMP machine, but I will try again. I once tried it with 200 disks and this was working well.
Kay
===== udev.c 1.74 vs edited =====
udev.c +13 -8
@@ -156,15 +156,17 @@ int main(int argc, char *argv[], char *e/* trigger timout to interrupt blocking syscalls */ alarm (ALARM_TIMEOUT ); -/* initialize udev database */ -if (udevdb_init (UDEVDB_DEFAULT ) !=0 ) -info ("error: unable to initialize database, continuing without database" ); -switch (act_type) {case UDEVSTART :dbg ("udevstart" );namedev_init (); + +/* add node for every device in sysfs, store everything in db and run scripts */ +if (udevdb_init (UDEVDB_DEFAULT ) !=0 ) +info ("error: unable to initialize database, continuing without database" ); retval =udev_start (); +udevdb_exit (); +break ;case ADD :dbg ("udev add" );
@@ -182,7 +184,10 @@ int main(int argc, char *argv[], char *enamedev_init ();/* name, create node, store in db */ +if (udevdb_init (UDEVDB_DEFAULT ) !=0 ) +info ("error: unable to initialize database, continuing without database" ); retval =udev_add_device (&udev, class_dev); +udevdb_exit ();/* run scripts */ dev_d_execute (&udev);
@@ -192,15 +197,15 @@ int main(int argc, char *argv[], char *ecase REMOVE :dbg ("udev remove" ); -/* get node from db, delete it*/ +/* get node from db, delete node, delete db entry */ +if (udevdb_init (UDEVDB_DEFAULT ) !=0 ) +info ("error: unable to initialize database, continuing without database" ); retval =udev_remove_device (&udev); +udevdb_exit ();/* run scripts */ dev_d_execute (&udev); } - -udevdb_exit (); - exit:logging_close ();return retval;
Quoted 30 lines · Kay Sievers
Quoted 15 lines · Greg KH
One of my coworkers (hi Nish) is trying to get 10000 disk support tested and working properly with linux and ran into a nasty udev issue yesterday. It seems that under a very high load, with a whole bunch of hotplug events happening (for when the disks get added to the system), udev hangs.
It hangs in the "grabbing the database lock" portion of the code (don't have actual logs of where it hangs, will try to get that next week.) But the interesting thing is we almost handle everything properly. udev creates the node, tries to write to the database. Then the timer expires and we report this. After the timer expires, udev is done for. It just sits and spins, doing a nanosleep constantly. Have 500 of those instances all running at once, all at a nice level of -10 is a sure way to bring a box (even a relatively big one) down hard.
Quoted 4 lines
So, while I'll agree finding the root tdb locking bug is a good idea, I wanted to point out that perhaps we should just exit udev if our timeout expires, instead of still waiting around. Or do you have a better solution?
Maybe the time the udev process locks the db is too long for that setup and the serialization of the concurrent locks will take longer than the timout of the process?
Here is a patch that tries to limit the access time to the db. The current udev opens the db, reads all the rules, writes to the db, executes the scripts and then closes the db. With this patch we open the db after the rules and close it directly after writing to it.
Quoted 8 lines
A rate limit in udevd may help here too, to keep that under control.
Quoted 3 lines
Try testing this out on your own using the
scsi_debugmodule and adding a few hundred disks. It also helps if you have scsi generic support enabled, as that creates 2 udev events for every disk created.I expect that this is completely different on a SMP machine, but I will try again. I once tried it with 200 disks and this was working well.
Here is another idea. I've ripped out the tdb code completely. We are maintaining a directory of files at /dev/.udevdb/* now. Every node will have a corresponding file (the slashes in DEVPATH are replaced by another char), that carries all neccessary data. The files are human readable:
[root@pim default]# ls -l /dev/.udevdb/*hda*
-rw-r--r-- 1 root root 26 Oct 24 06:32 /dev/.udevdb/block@hda -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda1 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda2 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda3 -rw-r--r-- 1 root root 54 Oct 24 06:32 /dev/.udevdb/block@hdc
[root@pim default]# cat /dev/.udevdb/block@hdc
P:/block/hdc
N:hdc
S:cdrom dvd cdwriter dvdwriter
A:0
This way we have _no_ locking at all in userspace, every event operation is completely independent from other events. No need to synchronize anything, which is expected to be much much faster and reliable on your 10.000 disk system. It may also work over NFS.
The patch is complete, but not well tested. It just stopped working on it, after the regression test was successful and my box was able to reboot and remove the symlinks grabbed form the files :)
Thanks,
Kay
Makefile +8 -16
@@ -113,11 +113,6 @@ CFLAGS := -pipe -# set up the proper tdb spinlock code if we can -ifeq ($(strip $(ARCH)),i386) - CFLAGS += -DUSE_SPINLOCKS -DINTEL_SPINLOCKS -endif - ifeq ($(strip $(USE_LOG)),true) CFLAGS += -DLOG endif
@@ -201,9 +196,6 @@ fi $(MAKE) -C klibc SUBDIRS=klibc -TDB = tdb/tdb.o \ - tdb/spinlock.o - SYSFS = $(PWD)/libsysfs/sysfs_bus.o \ $(PWD)/libsysfs/sysfs_class.o \ $(PWD)/libsysfs/sysfs_device.o \
@@ -221,8 +213,7 @@ namedev.o \ namedev_parse.o \ dev_d.o \ - $(SYSFS) \ - $(TDB) + $(SYSFS) HEADERS = udev.h \ udev_lib.h \
@@ -262,8 +253,8 @@ udev_version.h: @echo "Creating udev_version.h" @echo \#define UDEV_VERSION \"$(VERSION)\" > $@ - @echo \#define UDEV_ROOT \"$(udevdir)/\" >> $@ - @echo \#define UDEV_DB \"$(udevdir)/.udev.tdb\" >> $@ + @echo \#define UDEV_ROOT \"$(udevdir)\" >> $@ + @echo \#define UDEV_DB \"$(udevdir)/.udevdb\" >> $@ @echo \#define UDEV_CONFIG_DIR \"$(configdir)\" >> $@ @echo \#define UDEV_CONFIG_FILE \"$(configdir)/udev.conf\" >> $@ @echo \#define UDEV_RULES_FILE \"$(configdir)/rules.d\" >> $@
@@ -300,7 +291,7 @@ $(QUIET) $(STRIPCMD) $@ $(INFO): $(LIBC) $(INFO).o $(OBJS) $(HEADERS) - $(QUIET) $(LD) $(LDFLAGS) -o $@ $(CRT0) udevinfo.o udev_lib.o udev_config.o udevdb.o $(SYSFS) $(TDB) $(LIB_OBJS) $(ARCH_LIB_OBJS) + $(QUIET) $(LD) $(LDFLAGS) -o $@ $(CRT0) udevinfo.o udev_lib.o udev_config.o udevdb.o $(SYSFS) $(LIB_OBJS) $(ARCH_LIB_OBJS) $(QUIET) $(STRIPCMD) $@ $(DAEMON): $(LIBC) $(DAEMON).o $(OBJS) udevd.h
@@ -337,7 +328,8 @@ $(MAKE) -C klibc spotless -rm -f klibc/linux -DISTFILES = $(shell find . \( -not -name '.' \) -print | grep -v -e CVS -e "\.tar\.gz$" -e "\/\." -e releases -e BitKeeper -e SCCS -e "\.tdb$" -e test/sys | sort ) +DISTFILES = $(shell find . \( -not -name '.' \) -print | grep -v -e CVS -e "\.tar\.gz" -e "\/\." -e releases -e BitKeeper -e SCCS -e test/sys | sort ) +#DISTFILES = $(shell find . \( -not -name '.' \) -print | grep -v -e CVS -e "\.tar\.gz$" -e "\/\." -e releases -e BitKeeper -e SCCS -e "\.tdb$" -e test/sys | sort ) DISTDIR := $(RELEASE_NAME) srcdir = . release: clean
@@ -430,7 +422,7 @@ - ln -f -s $(sbindir)/$(WAIT) $(DESTDIR)$(hotplugdir)/05-wait_for_sysfs.hotplug ifndef DESTDIR - killall udevd - - rm -f $(udevdir)/.udev.tdb + - rm -f $(udevdir)/.udev/* endif @extras="$(EXTRAS)" ; for target in $$extras ; do \ echo $$target ; \
@@ -456,7 +448,7 @@ - rm $(usrbindir)/$(TESTER) - rm $(usrbindir)/$(WAIT) - rmdir $(hotplugdir) - - rm $(udevdir)/.udev.tdb + - rm -f $(udevdir)/.udev/* - rmdir $(udevdir) @extras="$(EXTRAS)" ; for target in $$extras ; do \ echo $$target ; \
dev_d.c +0 -1
@@ -47,7 +47,6 @@switch (pid) {case 0 :/* child */ -udevdb_exit ();/* close udevdb */ fd =open ("/dev/null" ,O_RDWR );if ( fd >=0 ) {dup2 (fd,STDOUT_FILENO );
etc/udev/udev.conf.in +4 -4
@@ -6,16 +6,16 @@ # udev_root - where in the filesystem to place the device nodes -udev_root="@udevdir@/" +udev_root="@udevdir@" # udev_db - The name and location of the udev database. -udev_db="@udevdir@/.udev.tdb" +udev_db="@udevdir@/.udevdb" # udev_rules - The name and location of the udev rules file -udev_rules="@configdir@/rules.d/" +udev_rules="@configdir@/rules.d" # udev_permissions - The name and location of the udev permission file -udev_permissions="@configdir@/permissions.d/" +udev_permissions="@configdir@/permissions.d" # default_mode - set the default mode for all nodes that have no # explicit match in the permissions file
namedev.c +21 -21
57 lines not on this page · raw message
test/udev-test.pl +3 -3
@@ -30,7 +30,7 @@ my $sysfs = "sys/"; my $udev_bin = "../udev"; my $udev_root = "udev-root/"; # !!! directory will be removed !!! -my $udev_db = ".udev.tdb"; +my $udev_db = ".udevdb"; my $perm = "udev.permissions"; my $main_conf = "udev-test.conf"; my $conf_tmp = "udev-test.rules";
@@ -1258,7 +1258,7 @@ } if (defined($config->{option}) && $config->{option} eq "clear") { - unlink($udev_db); + system("rm -rf $udev_db"); system("rm -rf $udev_root"); mkdir($udev_root) || die "unable to create udev_root: $udev_root\n"; }
@@ -1302,7 +1302,7 @@ print "$error errors occured\n\n"; # cleanup -unlink($udev_db); +system("rm -rf $udev_db"); system("rm -rf $udev_root"); unlink($conf_tmp); unlink($main_conf);
udev.8.in +2 -2
@@ -74,7 +74,7 @@ .TP .B udev_db The name and location of the udev database. The default value is -.IR @udevdir@/.udev.tdb . +.IR @udevdir@/.udevdb . .TP .B udev_rules The name of the udev rules file or directory to look for files with the suffix
@@ -117,7 +117,7 @@ udev_root="/udev" # udev_db - The name and location of the udev database -udev_db="/udev/.udev.tdb" +udev_db="/udev/.udevdb" # udev_rules - The name of the udev rules file or directory to look for files with the suffix .rules
udev.c +3 -12
@@ -38,8 +38,6 @@#include "namedev.h" #include "udevdb.h" -/* timeout flag for udevdb */ -extern sig_atomic_t gotalarm;/* global variables */ char **main_argv;
@@ -64,12 +62,11 @@ {switch (signum) {case SIGALRM : - gotalarm =1 ; -info ("error: timeout reached, event probably not handled correctly" ); -break ; +info ("we give up, timeout reached, event probably not handled correctly" ); +logging_close (); +exit (1 );case SIGINT :case SIGTERM : -udevdb_exit ();exit (20 + signum);default :dbg ("unhandled signal %d" , signum);
@@ -156,10 +153,6 @@/* trigger timout to interrupt blocking syscalls */ alarm (ALARM_TIMEOUT ); -/* initialize udev database */ -if (udevdb_init (UDEVDB_DEFAULT ) !=0 ) -info ("error: unable to initialize database, continuing without database" ); -switch (act_type) {case UDEVSTART :dbg ("udevstart" );
@@ -198,8 +191,6 @@/* run scripts */ dev_d_execute (&udev); } - -udevdb_exit (); exit:logging_close ();
udev.h +4 -9
@@ -41,26 +41,21 @@#define LINE_SIZE 256 -/* length of public data to store in udevdb */ -#define UDEVICE_DB_LEN (offsetof(struct udevice, devpath)) -struct udevice { +char devpath[DEVPATH_SIZE ]; +char subsystem[SUBSYSTEM_SIZE ];char name[NAME_SIZE ];char owner[OWNER_SIZE ];char group[GROUP_SIZE ];char type;int major;int minor; -unsigned int mode;/* not mode_t due to conflicting definitions in different libcs */ +mode_t mode;char symlink[NAME_SIZE ];int partitions;int config_line;char config_file[NAME_SIZE ];long config_uptime; - -/* private data, not stored in udevdb */ -char devpath[DEVPATH_SIZE ]; -char subsystem[SUBSYSTEM_SIZE ];char bus_id[SYSFS_NAME_LEN ];char bus[SYSFS_NAME_LEN ];char program_result[NAME_SIZE ];
@@ -81,7 +76,7 @@extern char **main_envp;extern char sysfs_path[SYSFS_PATH_MAX ];extern char udev_root[PATH_MAX ]; -extern char udev_db_filename[PATH_MAX +NAME_MAX ]; +extern char udev_db_path[PATH_MAX +NAME_MAX ];extern char udev_permissions_filename[PATH_MAX +NAME_MAX ];extern char udev_config_filename[PATH_MAX +NAME_MAX ];extern char udev_rules_filename[PATH_MAX +NAME_MAX ];
udev_add.c +10 -38
79 lines not on this page · raw message
udev_config.c +10 -9
@@ -42,7 +42,7 @@/* global variables */ char sysfs_path[SYSFS_PATH_MAX ];char udev_root[PATH_MAX ]; -char udev_db_filename[PATH_MAX +NAME_MAX ]; +char udev_db_path[PATH_MAX +NAME_MAX ];char udev_permissions_filename[PATH_MAX +NAME_MAX ];char udev_rules_filename[PATH_MAX +NAME_MAX ];char udev_config_filename[PATH_MAX +NAME_MAX ];
@@ -72,7 +72,7 @@* If any config values are specified, they will * override these values. */ strfieldcpy (udev_root,UDEV_ROOT ); -strfieldcpy (udev_db_filename,UDEV_DB ); +strfieldcpy (udev_db_path,UDEV_DB );strfieldcpy (udev_config_filename,UDEV_CONFIG_FILE );strfieldcpy (udev_rules_filename,UDEV_RULES_FILE );strfieldcpy (udev_permissions_filename,UDEV_PERMISSION_FILE );
@@ -181,24 +181,25 @@if (strcasecmp (variable,"udev_root" ) ==0 ) {strfieldcpy (udev_root, value); -leading_slash (udev_root); +no_trailing_slash (udev_root);continue ; }if (strcasecmp (variable,"udev_db" ) ==0 ) { -strfieldcpy (udev_db_filename, value); +strfieldcpy (udev_db_path, value); +no_trailing_slash (udev_db_path);continue ; }if (strcasecmp (variable,"udev_rules" ) ==0 ) {strfieldcpy (udev_rules_filename, value); -no_leading_slash (udev_rules_filename); +no_trailing_slash (udev_rules_filename);continue ; }if (strcasecmp (variable,"udev_permissions" ) ==0 ) {strfieldcpy (udev_permissions_filename, value); -no_leading_slash (udev_permissions_filename); +no_trailing_slash (udev_permissions_filename);continue ; }
@@ -244,7 +245,7 @@ temp =getenv ("SYSFS_PATH" );if (temp !=NULL ) {strfieldcpy (sysfs_path, temp); -no_leading_slash (sysfs_path); +no_trailing_slash (sysfs_path); } temp =getenv ("UDEV_CONFIG_FILE" );
@@ -255,7 +256,7 @@dbg ("sysfs_path='%s'" , sysfs_path);dbg_parse ("udev_root = %s" , udev_root);dbg_parse ("udev_config_filename = %s" , udev_config_filename); -dbg_parse ("udev_db_filename = %s" , udev_db_filename); +dbg_parse ("udev_db_path = %s" , udev_db_path);dbg_parse ("udev_rules_filename = %s" , udev_rules_filename);dbg_parse ("udev_permissions_filename = %s" , udev_permissions_filename);dbg_parse ("udev_log = %d" , udev_log);
@@ -264,7 +265,7 @@dbg ("udev_root = %s" , udev_root);dbg ("udev_config_filename = %s" , udev_config_filename); -dbg ("udev_db_filename = %s" , udev_db_filename); +dbg ("udev_db_path = %s" , udev_db_path);dbg ("udev_rules_filename = %s" , udev_rules_filename);dbg ("udev_permissions_filename = %s" , udev_permissions_filename);dbg ("udev_log_str = %d" , udev_log);
udev_lib.c +30 -15
63 lines not on this page · raw message
udev_lib.h +2 -2
@@ -83,11 +83,11 @@extern char *get_subsystem (char *subsystem);extern char get_device_type (const char *path,const char *subsystem);extern void udev_set_values (struct udevice *udev,const char * devpath,const char *subsystem); +extern int create_path (const char *path);extern int file_map (const char *filename,char **buf,size_t *bufsize);extern void file_unmap (char *buf,size_t bufsize);extern size_t buf_get_line (char *buf,size_t buflen,size_t cur); -extern void leading_slash (char *path); -extern void no_leading_slash (char *path); +extern void no_trailing_slash (char *path);extern int call_foreach_file (int fnct (char *f) ,char *filename,char *extension);extern int set_cloexec_flag (int desc,int value);
udev_remove.c +15 -18
@@ -102,10 +102,9 @@return retval; } -static int delete_node (struct udevice *dev) +static int delete_node (struct udevice *udev) {char filename[NAME_SIZE ]; -char linkname[NAME_SIZE ];char partitionname[NAME_SIZE ];int retval;int i;
@@ -113,8 +112,8 @@int len;int num; -strfieldcpy (filename, udev_root); -strfieldcat (filename, dev->name); +snprintf (filename,NAME_SIZE -1 ,"%s/%s" , udev_root, udev->name); + filename[NAME_SIZE -1 ] ='\0' ;info ("removing device node '%s'" , filename); retval =secure_unlink (filename);
@@ -122,7 +121,7 @@return retval;/* remove all_partitions nodes */ - num = dev->partitions; + num = udev->partitions;if (num >0 ) {info ("removing all_partitions '%s[1-%i]'" , filename, num);if (num >PARTITIONS_COUNT ) {
@@ -137,13 +136,15 @@ }/* remove subdirectories */ -if (strchr (dev->name,'/' )) +if (strchr (udev->name,'/' ))delete_path (filename); - foreach_strpart(dev->symlink, " ", pos, len) { + foreach_strpart(udev->symlink, " ", pos, len) { +char linkname[NAME_SIZE ]; +strfieldcpymax (linkname, pos, len+1 ); -strfieldcpy (filename, udev_root); -strfieldcat (filename, linkname); +snprintf (filename,NAME_SIZE -1 ,"%s/%s" , udev_root, linkname); + filename[NAME_SIZE -1 ] ='\0' ;dbg ("unlinking symlink '%s'" , filename); retval =unlink (filename);
@@ -154,7 +155,7 @@ filename, strerror(errno));return retval; } -if (strchr (dev->symlink,'/' )) { +if (strchr (udev->symlink,'/' )) {delete_path (filename); } }
@@ -168,18 +169,14 @@*/ int udev_remove_device (struct udevice *udev) { -struct udevice db_dev;const char *temp;int retval;if (udev->type !='b' && udev->type !='c' )return 0 ; - retval =udevdb_get_dev (udev->devpath, &db_dev); -if (retval ==0 ) { -/* copy over the stored values to our device */ -memcpy (udev, &db_dev,UDEVICE_DB_LEN ); - }else { + retval =udevdb_get_dev (udev); +if (retval) {/* fall back to kernel name */ temp =strrchr (udev->devpath,'/' );if (temp ==NULL )
@@ -189,10 +186,10 @@ }dbg ("remove name='%s'" , udev->name); -udevdb_delete_dev (udev->devpath); +udevdb_delete_dev (udev);/* use full path to the environment */ -snprintf (udev->devname,NAME_SIZE -1 ,"%s%s" , udev_root, udev->name); +snprintf (udev->devname,NAME_SIZE -1 ,"%s/%s" , udev_root, udev->name);return delete_node (udev); }
udevd.c +1 -1
@@ -147,7 +147,7 @@switch (pid) {case 0 :/* child */ -execle (udev_bin,"udev" , msg->subsystem,NULL , env); +execle (udev_bin, udev_bin, msg->subsystem,NULL , env);dbg ("exec of child failed" );_exit (1 );break ;
udevdb.c +132 -154
358 lines not on this page · raw message
udevdb.h +25 -13
@@ -1,22 +1,34 @@/* -* udevdb header file +* udevdb.h +* +* Userspace devfs +* +* Copyright (C) 2003 Greg Kroah-Hartman <[email protected]> +* Copyright (C) 2004 Kay Sievers <[email protected]> +* +* This program is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by the +* Free Software Foundation version 2 of the License. +* +* This program is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License along +* with this program; if not, write to the Free Software Foundation, Inc., +* 675 Mass Ave, Cambridge, MA 02139, USA. +* */ +#ifndef _UDEVDB_H_#define _UDEVDB_H_ -/* Udevdb initialization flags */ -#define UDEVDB_DEFAULT 0/* defaults database to use file */ -#define UDEVDB_INTERNAL 1/* don't store db on disk, use in memory */ - -/* function prototypes */ -extern void udevdb_exit (void ); -extern int udevdb_init (int init_flag); -extern int udevdb_open_ro (void ); -extern int udevdb_call_foreach (int (*user_record_handler) (const char *path,struct udevice *dev));extern int udevdb_add_dev (struct udevice *dev); -extern int udevdb_get_dev (const char *path,struct udevice *dev); -extern int udevdb_delete_dev (const char *path); -extern int udevdb_get_dev_byname (const char *name,char *path,struct udevice *dev); +extern int udevdb_get_dev (struct udevice *dev); +extern int udevdb_delete_dev (struct udevice *dev); + +extern int udevdb_get_dev_byname (struct udevice *udev,const char *name);#endif /* _UDEVDB_H_ */
udevinfo.8 +1 -4
@@ -29,7 +29,7 @@ .RB Needs " \-p " or " \-n " specified. .br Valid types are: -.BR name ", " symlink ", " mode " ," owner " , " group " , " path " or " all. +.BR name ", " symlink ", " path " or " all. .TP .BI \-p " sysfs_path" Specify the sysfs path of the device to query.
@@ -45,9 +45,6 @@ attributes along the device chain. Useful for finding unique attributes to compose a rule. .RB Needs " \-p " specified. -.TP -.B \-d -Dump the whole database. .TP .B \-h Print help text.
udevinfo.c +24 -73
185 lines not on this page · raw message
On Sun, Oct 24, Kay Sievers wrote:
this wastes one page per devnode, unless I misunderstand the concept of tmpfs.
Signature · 14 lines
USB is for mice, FireWire is for men! sUse lINUX ag, nÜRNBERG ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net [email protected] https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines · Olaf Hering
On Sun, Oct 24, Kay Sievers wrote:
this wastes one page per devnode, unless I misunderstand the concept of tmpfs.
This is right. The db files on tmpfs for an average box (~800 nodes) are taking ~3 MByte of swappable memory. If that's a problem, you can still configure a different location for the db directory.
Any better idea?
Kay
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 4 lines · Kay Sievers
Quoted 3 lines · Olaf Hering
On Sun, Oct 24, Kay Sievers wrote:
Quoted 8 lines
Quoted 2 lines
this wastes one page per devnode, unless I misunderstand the concept of tmpfs.
This is right. The db files on tmpfs for an average box (~800 nodes) are taking ~3 MByte of swappable memory. If that's a problem, you can still configure a different location for the db directory.
Any better idea?
I had not time yet to look at the udev locking bugs.
Signature · 14 lines
USB is for mice, FireWire is for men! sUse lINUX ag, nÜRNBERG ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net [email protected] https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 1 line · Olaf Hering
I had not time yet to look at the udev locking bugs.
I too would be unhappy if udev was wasting this much memory for extra files related to my /dev nodes (I moved to udev precisely to avoid problems like this <G>).
Not having looked at the code my suggestion may be way off-base, but wouldn't a very simple database that supports _only_ append operations (no in-place editing) suffice for udev's needs? There are many applications (BIND, Zope, etc.) that work just fine with databases like this, where any new or changed record is appended to the file, then pointers are updated to reflect the new record's presence. Seems to me that given the very small amount of update activity to udev's database (for most systems it will be 95+% appends when the system boots, and very little activity after that) that the small amount of wasted space due to "dead" records being in the database would be acceptable.
Then again, some of the threads about this issue seemed to relate to multiple systems sharing the same /dev directory over NFS, which I don't understand at all... Why would someone even want to do that (i.e. what would the value be)? If changes were made to handle non-locking-capable mounts for /dev, IMHO those changes should be reverted, and the documentation updated to say that if someone wants to use udev in such a situation, they are on their own :-)
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 7 lines · Kevin P. Fleming
Quoted 1 line · Olaf Hering
I had not time yet to look at the udev locking bugs.
I too would be unhappy if udev was wasting this much memory for extra files related to my /dev nodes (I moved to udev precisely to avoid problems like this <G>).
Just use a "normal" filesysten then and everything is fine.
Quoted 10 lines
Not having looked at the code my suggestion may be way off-base, but wouldn't a very simple database that supports _only_ append operations (no in-place editing) suffice for udev's needs? There are many applications (BIND, Zope, etc.) that work just fine with databases like this, where any new or changed record is appended to the file, then pointers are updated to reflect the new record's presence. Seems to me that given the very small amount of update activity to udev's database (for most systems it will be 95+% appends when the system boots, and very little activity after that) that the small amount of wasted space due to "dead" records being in the database would be acceptable.
What does "update pointers" mean? How do you manage the concurrent writing to this file if you have several thousand processes. What would _you_ do different from the tdb code? Please be more precise.
Quoted 7 lines
Then again, some of the threads about this issue seemed to relate to multiple systems sharing the same /dev directory over NFS, which I don't understand at all... Why would someone even want to do that (i.e. what would the value be)? If changes were made to handle non-locking-capable mounts for /dev, IMHO those changes should be reverted, and the documentation updated to say that if someone wants to use udev in such a situation, they are on their own :-)
Please read the code before making suggestions here.
Thanks,
Kay
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 1 line · Kay Sievers
Just use a "normal" filesysten then and everything is fine.
There are many good reasons for using tmpfs for /dev, and I would be very displeased if this will not be possible or practical anymore.
Signature · 13 lines
ciao, | Marco | [8716 riTQRdBOJ6m.A] ------------------------------------------------------- This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more http://productguide.itmanagersjournal.com/guidepromo.tmpl _______________________________________________ Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net [email protected] https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 5 lines · Marco d'Itri
Quoted 1 line · Kay Sievers
Just use a "normal" filesysten then and everything is fine.
There are many good reasons for using tmpfs for /dev, and I would be very displeased if this will not be possible or practical anymore.
Yes it's nice and there is still nothing wrong with tmpfs use. It's just a bit more space needed than before. Ever thought about much memory _every_ file in sysfs takes?
Kay
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 58 lines · Kay Sievers
Quoted 30 lines · Kay Sievers
Quoted 15 lines · Greg KH
One of my coworkers (hi Nish) is trying to get 10000 disk support tested and working properly with linux and ran into a nasty udev issue yesterday. It seems that under a very high load, with a whole bunch of hotplug events happening (for when the disks get added to the system), udev hangs.
It hangs in the "grabbing the database lock" portion of the code (don't have actual logs of where it hangs, will try to get that next week.) But the interesting thing is we almost handle everything properly. udev creates the node, tries to write to the database. Then the timer expires and we report this. After the timer expires, udev is done for. It just sits and spins, doing a nanosleep constantly. Have 500 of those instances all running at once, all at a nice level of -10 is a sure way to bring a box (even a relatively big one) down hard.
Quoted 4 lines
So, while I'll agree finding the root tdb locking bug is a good idea, I wanted to point out that perhaps we should just exit udev if our timeout expires, instead of still waiting around. Or do you have a better solution?
Maybe the time the udev process locks the db is too long for that setup and the serialization of the concurrent locks will take longer than the timout of the process?
Here is a patch that tries to limit the access time to the db. The current udev opens the db, reads all the rules, writes to the db, executes the scripts and then closes the db. With this patch we open the db after the rules and close it directly after writing to it.
Quoted 8 lines
A rate limit in udevd may help here too, to keep that under control.
Quoted 3 lines
Try testing this out on your own using the
scsi_debugmodule and adding a few hundred disks. It also helps if you have scsi generic support enabled, as that creates 2 udev events for every disk created.I expect that this is completely different on a SMP machine, but I will try again. I once tried it with 200 disks and this was working well.
Here is another idea. I've ripped out the tdb code completely. We are maintaining a directory of files at
/dev/.udevdb/*now. Every node will have a corresponding file (the slashes in DEVPATH are replaced by another char), that carries all neccessary data. The files are human readable:[root@pim default]# ls -l
/dev/.udevdb/*hda*-rw-r--r-- 1 root root 26 Oct 24 06:32 /dev/.udevdb/block@hda -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda1 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda2 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda3 -rw-r--r-- 1 root root 54 Oct 24 06:32 /dev/.udevdb/block@hdc[root@pim default]# cat
/dev/.udevdb/block@hdc
P:/block/hdc
N:hdc
S:cdrom dvd cdwriter dvdwriter
A:0
Quoted 8 lines
This way we have _no_ locking at all in userspace, every event operation is completely independent from other events. No need to synchronize anything, which is expected to be much much faster and reliable on your 10.000 disk system. It may also work over NFS.
The patch is complete, but not well tested. It just stopped working on it, after the regression test was successful and my box was able to reboot and remove the symlinks grabbed form the files :)
New patch with a fix for a bug introduced in udevinfo.
Thanks,
Kay
Makefile +8 -16
@@ -113,11 +113,6 @@ CFLAGS := -pipe -# set up the proper tdb spinlock code if we can -ifeq ($(strip $(ARCH)),i386) - CFLAGS += -DUSE_SPINLOCKS -DINTEL_SPINLOCKS -endif - ifeq ($(strip $(USE_LOG)),true) CFLAGS += -DLOG endif
@@ -201,9 +196,6 @@ fi $(MAKE) -C klibc SUBDIRS=klibc -TDB = tdb/tdb.o \ - tdb/spinlock.o - SYSFS = $(PWD)/libsysfs/sysfs_bus.o \ $(PWD)/libsysfs/sysfs_class.o \ $(PWD)/libsysfs/sysfs_device.o \
@@ -221,8 +213,7 @@ namedev.o \ namedev_parse.o \ dev_d.o \ - $(SYSFS) \ - $(TDB) + $(SYSFS) HEADERS = udev.h \ udev_lib.h \
@@ -262,8 +253,8 @@ udev_version.h: @echo "Creating udev_version.h" @echo \#define UDEV_VERSION \"$(VERSION)\" > $@ - @echo \#define UDEV_ROOT \"$(udevdir)/\" >> $@ - @echo \#define UDEV_DB \"$(udevdir)/.udev.tdb\" >> $@ + @echo \#define UDEV_ROOT \"$(udevdir)\" >> $@ + @echo \#define UDEV_DB \"$(udevdir)/.udevdb\" >> $@ @echo \#define UDEV_CONFIG_DIR \"$(configdir)\" >> $@ @echo \#define UDEV_CONFIG_FILE \"$(configdir)/udev.conf\" >> $@ @echo \#define UDEV_RULES_FILE \"$(configdir)/rules.d\" >> $@
@@ -300,7 +291,7 @@ $(QUIET) $(STRIPCMD) $@ $(INFO): $(LIBC) $(INFO).o $(OBJS) $(HEADERS) - $(QUIET) $(LD) $(LDFLAGS) -o $@ $(CRT0) udevinfo.o udev_lib.o udev_config.o udevdb.o $(SYSFS) $(TDB) $(LIB_OBJS) $(ARCH_LIB_OBJS) + $(QUIET) $(LD) $(LDFLAGS) -o $@ $(CRT0) udevinfo.o udev_lib.o udev_config.o udevdb.o $(SYSFS) $(LIB_OBJS) $(ARCH_LIB_OBJS) $(QUIET) $(STRIPCMD) $@ $(DAEMON): $(LIBC) $(DAEMON).o $(OBJS) udevd.h
@@ -337,7 +328,8 @@ $(MAKE) -C klibc spotless -rm -f klibc/linux -DISTFILES = $(shell find . \( -not -name '.' \) -print | grep -v -e CVS -e "\.tar\.gz$" -e "\/\." -e releases -e BitKeeper -e SCCS -e "\.tdb$" -e test/sys | sort ) +DISTFILES = $(shell find . \( -not -name '.' \) -print | grep -v -e CVS -e "\.tar\.gz" -e "\/\." -e releases -e BitKeeper -e SCCS -e test/sys | sort ) +#DISTFILES = $(shell find . \( -not -name '.' \) -print | grep -v -e CVS -e "\.tar\.gz$" -e "\/\." -e releases -e BitKeeper -e SCCS -e "\.tdb$" -e test/sys | sort ) DISTDIR := $(RELEASE_NAME) srcdir = . release: clean
@@ -430,7 +422,7 @@ - ln -f -s $(sbindir)/$(WAIT) $(DESTDIR)$(hotplugdir)/05-wait_for_sysfs.hotplug ifndef DESTDIR - killall udevd - - rm -f $(udevdir)/.udev.tdb + - rm -f $(udevdir)/.udev/* endif @extras="$(EXTRAS)" ; for target in $$extras ; do \ echo $$target ; \
@@ -456,7 +448,7 @@ - rm $(usrbindir)/$(TESTER) - rm $(usrbindir)/$(WAIT) - rmdir $(hotplugdir) - - rm $(udevdir)/.udev.tdb + - rm -f $(udevdir)/.udev/* - rmdir $(udevdir) @extras="$(EXTRAS)" ; for target in $$extras ; do \ echo $$target ; \
dev_d.c +0 -1
@@ -47,7 +47,6 @@switch (pid) {case 0 :/* child */ -udevdb_exit ();/* close udevdb */ fd =open ("/dev/null" ,O_RDWR );if ( fd >=0 ) {dup2 (fd,STDOUT_FILENO );
etc/udev/udev.conf.in +4 -4
@@ -6,16 +6,16 @@ # udev_root - where in the filesystem to place the device nodes -udev_root="@udevdir@/" +udev_root="@udevdir@" # udev_db - The name and location of the udev database. -udev_db="@udevdir@/.udev.tdb" +udev_db="@udevdir@/.udevdb" # udev_rules - The name and location of the udev rules file -udev_rules="@configdir@/rules.d/" +udev_rules="@configdir@/rules.d" # udev_permissions - The name and location of the udev permission file -udev_permissions="@configdir@/permissions.d/" +udev_permissions="@configdir@/permissions.d" # default_mode - set the default mode for all nodes that have no # explicit match in the permissions file
namedev.c +21 -21
57 lines not on this page · raw message
test/udev-test.pl +3 -3
@@ -30,7 +30,7 @@ my $sysfs = "sys/"; my $udev_bin = "../udev"; my $udev_root = "udev-root/"; # !!! directory will be removed !!! -my $udev_db = ".udev.tdb"; +my $udev_db = ".udevdb"; my $perm = "udev.permissions"; my $main_conf = "udev-test.conf"; my $conf_tmp = "udev-test.rules";
@@ -1258,7 +1258,7 @@ } if (defined($config->{option}) && $config->{option} eq "clear") { - unlink($udev_db); + system("rm -rf $udev_db"); system("rm -rf $udev_root"); mkdir($udev_root) || die "unable to create udev_root: $udev_root\n"; }
@@ -1302,7 +1302,7 @@ print "$error errors occured\n\n"; # cleanup -unlink($udev_db); +system("rm -rf $udev_db"); system("rm -rf $udev_root"); unlink($conf_tmp); unlink($main_conf);
udev.8.in +2 -2
@@ -74,7 +74,7 @@ .TP .B udev_db The name and location of the udev database. The default value is -.IR @udevdir@/.udev.tdb . +.IR @udevdir@/.udevdb . .TP .B udev_rules The name of the udev rules file or directory to look for files with the suffix
@@ -117,7 +117,7 @@ udev_root="/udev" # udev_db - The name and location of the udev database -udev_db="/udev/.udev.tdb" +udev_db="/udev/.udevdb" # udev_rules - The name of the udev rules file or directory to look for files with the suffix .rules
udev.c +3 -12
@@ -38,8 +38,6 @@#include "namedev.h" #include "udevdb.h" -/* timeout flag for udevdb */ -extern sig_atomic_t gotalarm;/* global variables */ char **main_argv;
@@ -64,12 +62,11 @@ {switch (signum) {case SIGALRM : - gotalarm =1 ; -info ("error: timeout reached, event probably not handled correctly" ); -break ; +info ("we give up, timeout reached, event probably not handled correctly" ); +logging_close (); +exit (1 );case SIGINT :case SIGTERM : -udevdb_exit ();exit (20 + signum);default :dbg ("unhandled signal %d" , signum);
@@ -156,10 +153,6 @@/* trigger timout to interrupt blocking syscalls */ alarm (ALARM_TIMEOUT ); -/* initialize udev database */ -if (udevdb_init (UDEVDB_DEFAULT ) !=0 ) -info ("error: unable to initialize database, continuing without database" ); -switch (act_type) {case UDEVSTART :dbg ("udevstart" );
@@ -198,8 +191,6 @@/* run scripts */ dev_d_execute (&udev); } - -udevdb_exit (); exit:logging_close ();
udev.h +4 -9
@@ -41,26 +41,21 @@#define LINE_SIZE 256 -/* length of public data to store in udevdb */ -#define UDEVICE_DB_LEN (offsetof(struct udevice, devpath)) -struct udevice { +char devpath[DEVPATH_SIZE ]; +char subsystem[SUBSYSTEM_SIZE ];char name[NAME_SIZE ];char owner[OWNER_SIZE ];char group[GROUP_SIZE ];char type;int major;int minor; -unsigned int mode;/* not mode_t due to conflicting definitions in different libcs */ +mode_t mode;char symlink[NAME_SIZE ];int partitions;int config_line;char config_file[NAME_SIZE ];long config_uptime; - -/* private data, not stored in udevdb */ -char devpath[DEVPATH_SIZE ]; -char subsystem[SUBSYSTEM_SIZE ];char bus_id[SYSFS_NAME_LEN ];char bus[SYSFS_NAME_LEN ];char program_result[NAME_SIZE ];
@@ -81,7 +76,7 @@extern char **main_envp;extern char sysfs_path[SYSFS_PATH_MAX ];extern char udev_root[PATH_MAX ]; -extern char udev_db_filename[PATH_MAX +NAME_MAX ]; +extern char udev_db_path[PATH_MAX +NAME_MAX ];extern char udev_permissions_filename[PATH_MAX +NAME_MAX ];extern char udev_config_filename[PATH_MAX +NAME_MAX ];extern char udev_rules_filename[PATH_MAX +NAME_MAX ];
udev_add.c +10 -38
79 lines not on this page · raw message
udev_config.c +10 -9
@@ -42,7 +42,7 @@/* global variables */ char sysfs_path[SYSFS_PATH_MAX ];char udev_root[PATH_MAX ]; -char udev_db_filename[PATH_MAX +NAME_MAX ]; +char udev_db_path[PATH_MAX +NAME_MAX ];char udev_permissions_filename[PATH_MAX +NAME_MAX ];char udev_rules_filename[PATH_MAX +NAME_MAX ];char udev_config_filename[PATH_MAX +NAME_MAX ];
@@ -72,7 +72,7 @@* If any config values are specified, they will * override these values. */ strfieldcpy (udev_root,UDEV_ROOT ); -strfieldcpy (udev_db_filename,UDEV_DB ); +strfieldcpy (udev_db_path,UDEV_DB );strfieldcpy (udev_config_filename,UDEV_CONFIG_FILE );strfieldcpy (udev_rules_filename,UDEV_RULES_FILE );strfieldcpy (udev_permissions_filename,UDEV_PERMISSION_FILE );
@@ -181,24 +181,25 @@if (strcasecmp (variable,"udev_root" ) ==0 ) {strfieldcpy (udev_root, value); -leading_slash (udev_root); +no_trailing_slash (udev_root);continue ; }if (strcasecmp (variable,"udev_db" ) ==0 ) { -strfieldcpy (udev_db_filename, value); +strfieldcpy (udev_db_path, value); +no_trailing_slash (udev_db_path);continue ; }if (strcasecmp (variable,"udev_rules" ) ==0 ) {strfieldcpy (udev_rules_filename, value); -no_leading_slash (udev_rules_filename); +no_trailing_slash (udev_rules_filename);continue ; }if (strcasecmp (variable,"udev_permissions" ) ==0 ) {strfieldcpy (udev_permissions_filename, value); -no_leading_slash (udev_permissions_filename); +no_trailing_slash (udev_permissions_filename);continue ; }
@@ -244,7 +245,7 @@ temp =getenv ("SYSFS_PATH" );if (temp !=NULL ) {strfieldcpy (sysfs_path, temp); -no_leading_slash (sysfs_path); +no_trailing_slash (sysfs_path); } temp =getenv ("UDEV_CONFIG_FILE" );
@@ -255,7 +256,7 @@dbg ("sysfs_path='%s'" , sysfs_path);dbg_parse ("udev_root = %s" , udev_root);dbg_parse ("udev_config_filename = %s" , udev_config_filename); -dbg_parse ("udev_db_filename = %s" , udev_db_filename); +dbg_parse ("udev_db_path = %s" , udev_db_path);dbg_parse ("udev_rules_filename = %s" , udev_rules_filename);dbg_parse ("udev_permissions_filename = %s" , udev_permissions_filename);dbg_parse ("udev_log = %d" , udev_log);
@@ -264,7 +265,7 @@dbg ("udev_root = %s" , udev_root);dbg ("udev_config_filename = %s" , udev_config_filename); -dbg ("udev_db_filename = %s" , udev_db_filename); +dbg ("udev_db_path = %s" , udev_db_path);dbg ("udev_rules_filename = %s" , udev_rules_filename);dbg ("udev_permissions_filename = %s" , udev_permissions_filename);dbg ("udev_log_str = %d" , udev_log);
udev_lib.c +30 -15
63 lines not on this page · raw message
udev_lib.h +2 -2
@@ -83,11 +83,11 @@extern char *get_subsystem (char *subsystem);extern char get_device_type (const char *path,const char *subsystem);extern void udev_set_values (struct udevice *udev,const char * devpath,const char *subsystem); +extern int create_path (const char *path);extern int file_map (const char *filename,char **buf,size_t *bufsize);extern void file_unmap (char *buf,size_t bufsize);extern size_t buf_get_line (char *buf,size_t buflen,size_t cur); -extern void leading_slash (char *path); -extern void no_leading_slash (char *path); +extern void no_trailing_slash (char *path);extern int call_foreach_file (int fnct (char *f) ,char *filename,char *extension);extern int set_cloexec_flag (int desc,int value);
udev_remove.c +15 -18
@@ -102,10 +102,9 @@return retval; } -static int delete_node (struct udevice *dev) +static int delete_node (struct udevice *udev) {char filename[NAME_SIZE ]; -char linkname[NAME_SIZE ];char partitionname[NAME_SIZE ];int retval;int i;
@@ -113,8 +112,8 @@int len;int num; -strfieldcpy (filename, udev_root); -strfieldcat (filename, dev->name); +snprintf (filename,NAME_SIZE -1 ,"%s/%s" , udev_root, udev->name); + filename[NAME_SIZE -1 ] ='\0' ;info ("removing device node '%s'" , filename); retval =secure_unlink (filename);
@@ -122,7 +121,7 @@return retval;/* remove all_partitions nodes */ - num = dev->partitions; + num = udev->partitions;if (num >0 ) {info ("removing all_partitions '%s[1-%i]'" , filename, num);if (num >PARTITIONS_COUNT ) {
@@ -137,13 +136,15 @@ }/* remove subdirectories */ -if (strchr (dev->name,'/' )) +if (strchr (udev->name,'/' ))delete_path (filename); - foreach_strpart(dev->symlink, " ", pos, len) { + foreach_strpart(udev->symlink, " ", pos, len) { +char linkname[NAME_SIZE ]; +strfieldcpymax (linkname, pos, len+1 ); -strfieldcpy (filename, udev_root); -strfieldcat (filename, linkname); +snprintf (filename,NAME_SIZE -1 ,"%s/%s" , udev_root, linkname); + filename[NAME_SIZE -1 ] ='\0' ;dbg ("unlinking symlink '%s'" , filename); retval =unlink (filename);
@@ -154,7 +155,7 @@ filename, strerror(errno));return retval; } -if (strchr (dev->symlink,'/' )) { +if (strchr (udev->symlink,'/' )) {delete_path (filename); } }
@@ -168,18 +169,14 @@*/ int udev_remove_device (struct udevice *udev) { -struct udevice db_dev;const char *temp;int retval;if (udev->type !='b' && udev->type !='c' )return 0 ; - retval =udevdb_get_dev (udev->devpath, &db_dev); -if (retval ==0 ) { -/* copy over the stored values to our device */ -memcpy (udev, &db_dev,UDEVICE_DB_LEN ); - }else { + retval =udevdb_get_dev (udev); +if (retval) {/* fall back to kernel name */ temp =strrchr (udev->devpath,'/' );if (temp ==NULL )
@@ -189,10 +186,10 @@ }dbg ("remove name='%s'" , udev->name); -udevdb_delete_dev (udev->devpath); +udevdb_delete_dev (udev);/* use full path to the environment */ -snprintf (udev->devname,NAME_SIZE -1 ,"%s%s" , udev_root, udev->name); +snprintf (udev->devname,NAME_SIZE -1 ,"%s/%s" , udev_root, udev->name);return delete_node (udev); }
udevd.c +1 -1
@@ -147,7 +147,7 @@switch (pid) {case 0 :/* child */ -execle (udev_bin,"udev" , msg->subsystem,NULL , env); +execle (udev_bin, udev_bin, msg->subsystem,NULL , env);dbg ("exec of child failed" );_exit (1 );break ;
udevdb.c +132 -154
358 lines not on this page · raw message
udevdb.h +25 -13
@@ -1,22 +1,34 @@/* -* udevdb header file +* udevdb.h +* +* Userspace devfs +* +* Copyright (C) 2003 Greg Kroah-Hartman <[email protected]> +* Copyright (C) 2004 Kay Sievers <[email protected]> +* +* This program is free software; you can redistribute it and/or modify it +* under the terms of the GNU General Public License as published by the +* Free Software Foundation version 2 of the License. +* +* This program is distributed in the hope that it will be useful, but +* WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +* General Public License for more details. +* +* You should have received a copy of the GNU General Public License along +* with this program; if not, write to the Free Software Foundation, Inc., +* 675 Mass Ave, Cambridge, MA 02139, USA. +* */ +#ifndef _UDEVDB_H_#define _UDEVDB_H_ -/* Udevdb initialization flags */ -#define UDEVDB_DEFAULT 0/* defaults database to use file */ -#define UDEVDB_INTERNAL 1/* don't store db on disk, use in memory */ - -/* function prototypes */ -extern void udevdb_exit (void ); -extern int udevdb_init (int init_flag); -extern int udevdb_open_ro (void ); -extern int udevdb_call_foreach (int (*user_record_handler) (const char *path,struct udevice *dev));extern int udevdb_add_dev (struct udevice *dev); -extern int udevdb_get_dev (const char *path,struct udevice *dev); -extern int udevdb_delete_dev (const char *path); -extern int udevdb_get_dev_byname (const char *name,char *path,struct udevice *dev); +extern int udevdb_get_dev (struct udevice *dev); +extern int udevdb_delete_dev (struct udevice *dev); + +extern int udevdb_get_dev_byname (struct udevice *udev,const char *name);#endif /* _UDEVDB_H_ */
udevinfo.8 +1 -4
@@ -29,7 +29,7 @@ .RB Needs " \-p " or " \-n " specified. .br Valid types are: -.BR name ", " symlink ", " mode " ," owner " , " group " , " path " or " all. +.BR name ", " symlink ", " path " or " all. .TP .BI \-p " sysfs_path" Specify the sysfs path of the device to query.
@@ -45,9 +45,6 @@ attributes along the device chain. Useful for finding unique attributes to compose a rule. .RB Needs " \-p " specified. -.TP -.B \-d -Dump the whole database. .TP .B \-h Print help text.
udevinfo.c +25 -73
186 lines not on this page · raw message
Quoted 10 lines · Kay Sievers
Quoted 5 lines · Marco d'Itri
Quoted 1 line · Kay Sievers
Just use a "normal" filesysten then and everything is fine.
There are many good reasons for using tmpfs for /dev, and I would be very displeased if this will not be possible or practical anymore.
Yes it's nice and there is still nothing wrong with tmpfs use. It's just a bit more space needed than before. Ever thought about much memory _every_ file in sysfs takes?
Oh yeah, a number of us are _very_ aware of that :)
So much so that the -mm tree has a patch that reduces that very memory footprint, and those patches are trying valiantly to make it into the mainline kernel tree.
That being said, putting a single, small (less than a single page) file for every device node in the system in tmpfs should not be a big deal. Anyone have any real numbers that they are worried about?
thanks,
greg k-h
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 8 lines · Greg KH
Quoted 7 lines · Kay Sievers
Quoted 5 lines · Marco d'Itri
Quoted 1 line · Kay Sievers
Just use a "normal" filesysten then and everything is fine.
There are many good reasons for using tmpfs for /dev, and I would be very displeased if this will not be possible or practical anymore.
Quoted 6 lines
Quoted 3 lines
Yes it's nice and there is still nothing wrong with tmpfs use. It's just a bit more space needed than before. Ever thought about much memory _every_ file in sysfs takes?
That being said, putting a single, small (less than a single page) file for every device node in the system in tmpfs should not be a big deal. Anyone have any real numbers that they are worried about?
On my box with app. 800 device nodes (which should be the number of app. 90 percent of the boxes out there) tmpfs is ~3 MByte big with the files db. That's not cool, but it's a limitation of tmpfs not udev :)
I personally think the complete independend and reliable operation of concurrent events is more worth than to work around a filesystem limitation with our own single-file-db implementation.
The average multi-file-db file is less than 40 Bytes long now. Maybe it will grow a bit, but not much, as we only need to store the names of our created nodes and the all_partitions count. Every other information (major/minor, type) can be stat()'ed from the node itself, we don't need to store it in the db.
The question is: Can we change the tmpfs implementation to store file content inline if it fits into struct shmem_inode_info. Mostly the same way symlink targets are handled today (only targets longer than 110 bytes are allocating a page).
This would be much more efficient, uses minimal memory and is the fastest we can get. And it would solve all the problems, I don't have :)
Thanks,
Kay
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 3 lines · Kay Sievers
The question is: Can we change the tmpfs implementation to store file content inline if it fits into struct
shmem_inode_info. Mostly the same way symlink targets are handled today (only targets longer than 110 bytes are allocating a page).
Now THAT would be a slick solution! It solves this problem, and is a win for everyone else using tmpfs as well.
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 64 lines · Kay Sievers
Quoted 63 lines · Kay Sievers
Quoted 30 lines · Kay Sievers
Quoted 15 lines · Greg KH
One of my coworkers (hi Nish) is trying to get 10000 disk support tested and working properly with linux and ran into a nasty udev issue yesterday. It seems that under a very high load, with a whole bunch of hotplug events happening (for when the disks get added to the system), udev hangs.
It hangs in the "grabbing the database lock" portion of the code (don't have actual logs of where it hangs, will try to get that next week.) But the interesting thing is we almost handle everything properly. udev creates the node, tries to write to the database. Then the timer expires and we report this. After the timer expires, udev is done for. It just sits and spins, doing a nanosleep constantly. Have 500 of those instances all running at once, all at a nice level of -10 is a sure way to bring a box (even a relatively big one) down hard.
Quoted 4 lines
So, while I'll agree finding the root tdb locking bug is a good idea, I wanted to point out that perhaps we should just exit udev if our timeout expires, instead of still waiting around. Or do you have a better solution?
Maybe the time the udev process locks the db is too long for that setup and the serialization of the concurrent locks will take longer than the timout of the process?
Here is a patch that tries to limit the access time to the db. The current udev opens the db, reads all the rules, writes to the db, executes the scripts and then closes the db. With this patch we open the db after the rules and close it directly after writing to it.
Quoted 8 lines
A rate limit in udevd may help here too, to keep that under control.
Quoted 3 lines
Try testing this out on your own using the
scsi_debugmodule and adding a few hundred disks. It also helps if you have scsi generic support enabled, as that creates 2 udev events for every disk created.I expect that this is completely different on a SMP machine, but I will try again. I once tried it with 200 disks and this was working well.
Here is another idea. I've ripped out the tdb code completely. We are maintaining a directory of files at
/dev/.udevdb/*now. Every node will have a corresponding file (the slashes in DEVPATH are replaced by another char), that carries all neccessary data. The files are human readable:[root@pim default]# ls -l
/dev/.udevdb/*hda*-rw-r--r-- 1 root root 26 Oct 24 06:32 /dev/.udevdb/block@hda -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda1 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda2 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda3 -rw-r--r-- 1 root root 54 Oct 24 06:32 /dev/.udevdb/block@hdc[root@pim default]# cat
/dev/.udevdb/block@hdc
P:/block/hdc
N:hdc
S:cdrom dvd cdwriter dvdwriter
A:0This way we have _no_ locking at all in userspace, every event operation is completely independent from other events. No need to synchronize anything, which is expected to be much much faster and reliable on your 10.000 disk system. It may also work over NFS.
Quoted 5 lines
Quoted 3 lines
The patch is complete, but not well tested. It just stopped working on it, after the regression test was successful and my box was able to reboot and remove the symlinks grabbed form the files :)
New patch with a fix for a bug introduced in udevinfo.
I am going to rerun my test with this patch applied. I will respond soon with the results.
-Nish
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 65 lines · Nishanth Aravamudan
Quoted 64 lines · Kay Sievers
Quoted 63 lines · Kay Sievers
Quoted 30 lines · Kay Sievers
Quoted 15 lines · Greg KH
One of my coworkers (hi Nish) is trying to get 10000 disk support tested and working properly with linux and ran into a nasty udev issue yesterday. It seems that under a very high load, with a whole bunch of hotplug events happening (for when the disks get added to the system), udev hangs.
It hangs in the "grabbing the database lock" portion of the code (don't have actual logs of where it hangs, will try to get that next week.) But the interesting thing is we almost handle everything properly. udev creates the node, tries to write to the database. Then the timer expires and we report this. After the timer expires, udev is done for. It just sits and spins, doing a nanosleep constantly. Have 500 of those instances all running at once, all at a nice level of -10 is a sure way to bring a box (even a relatively big one) down hard.
Quoted 4 lines
So, while I'll agree finding the root tdb locking bug is a good idea, I wanted to point out that perhaps we should just exit udev if our timeout expires, instead of still waiting around. Or do you have a better solution?
Maybe the time the udev process locks the db is too long for that setup and the serialization of the concurrent locks will take longer than the timout of the process?
Here is a patch that tries to limit the access time to the db. The current udev opens the db, reads all the rules, writes to the db, executes the scripts and then closes the db. With this patch we open the db after the rules and close it directly after writing to it.
Quoted 8 lines
A rate limit in udevd may help here too, to keep that under control.
Quoted 3 lines
Try testing this out on your own using the
scsi_debugmodule and adding a few hundred disks. It also helps if you have scsi generic support enabled, as that creates 2 udev events for every disk created.I expect that this is completely different on a SMP machine, but I will try again. I once tried it with 200 disks and this was working well.
Here is another idea. I've ripped out the tdb code completely. We are maintaining a directory of files at
/dev/.udevdb/*now. Every node will have a corresponding file (the slashes in DEVPATH are replaced by another char), that carries all neccessary data. The files are human readable:[root@pim default]# ls -l
/dev/.udevdb/*hda*-rw-r--r-- 1 root root 26 Oct 24 06:32 /dev/.udevdb/block@hda -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda1 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda2 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda3 -rw-r--r-- 1 root root 54 Oct 24 06:32 /dev/.udevdb/block@hdc[root@pim default]# cat
/dev/.udevdb/block@hdc
P:/block/hdc
N:hdc
S:cdrom dvd cdwriter dvdwriter
A:0This way we have _no_ locking at all in userspace, every event operation is completely independent from other events. No need to synchronize anything, which is expected to be much much faster and reliable on your 10.000 disk system. It may also work over NFS.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
The patch is complete, but not well tested. It just stopped working on it, after the regression test was successful and my box was able to reboot and remove the symlinks grabbed form the files :)
New patch with a fix for a bug introduced in udevinfo.
I am going to rerun my test with this patch applied. I will respond soon with the results.
Nice. It was just a bug with the -r option in udevinfo. If you don't use this db-query program, nothing has changed.
Are you able to hotplug-remove the disks from the system? It would be nice to know, if the nodes and symlinks are all correctly removed if you use custom names, given by a rule for your thousands of disks setup.
Thanks,
Kay
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
Quoted 70 lines · Kay Sievers
Quoted 69 lines · Nishanth Aravamudan
Quoted 68 lines · Kay Sievers
Quoted 66 lines · Kay Sievers
Quoted 30 lines · Kay Sievers
Quoted 15 lines · Greg KH
One of my coworkers (hi Nish) is trying to get 10000 disk support tested and working properly with linux and ran into a nasty udev issue yesterday. It seems that under a very high load, with a whole bunch of hotplug events happening (for when the disks get added to the system), udev hangs.
It hangs in the "grabbing the database lock" portion of the code (don't have actual logs of where it hangs, will try to get that next week.) But the interesting thing is we almost handle everything properly. udev creates the node, tries to write to the database. Then the timer expires and we report this. After the timer expires, udev is done for. It just sits and spins, doing a nanosleep constantly. Have 500 of those instances all running at once, all at a nice level of -10 is a sure way to bring a box (even a relatively big one) down hard.
Quoted 4 lines
So, while I'll agree finding the root tdb locking bug is a good idea, I wanted to point out that perhaps we should just exit udev if our timeout expires, instead of still waiting around. Or do you have a better solution?
Maybe the time the udev process locks the db is too long for that setup and the serialization of the concurrent locks will take longer than the timout of the process?
Here is a patch that tries to limit the access time to the db. The current udev opens the db, reads all the rules, writes to the db, executes the scripts and then closes the db. With this patch we open the db after the rules and close it directly after writing to it.
Quoted 8 lines
A rate limit in udevd may help here too, to keep that under control.
Quoted 3 lines
Try testing this out on your own using the
scsi_debugmodule and adding a few hundred disks. It also helps if you have scsi generic support enabled, as that creates 2 udev events for every disk created.I expect that this is completely different on a SMP machine, but I will try again. I once tried it with 200 disks and this was working well.
Here is another idea. I've ripped out the tdb code completely. We are maintaining a directory of files at
/dev/.udevdb/*now. Every node will have a corresponding file (the slashes in DEVPATH are replaced by another char), that carries all neccessary data. The files are human readable:[root@pim default]# ls -l
/dev/.udevdb/*hda*-rw-r--r-- 1 root root 26 Oct 24 06:32 /dev/.udevdb/block@hda -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda1 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda2 -rw-r--r-- 1 root root 32 Oct 24 06:32 /dev/.udevdb/block@hda@hda3 -rw-r--r-- 1 root root 54 Oct 24 06:32 /dev/.udevdb/block@hdc[root@pim default]# cat
/dev/.udevdb/block@hdc
P:/block/hdc
N:hdc
S:cdrom dvd cdwriter dvdwriter
A:0This way we have _no_ locking at all in userspace, every event operation is completely independent from other events. No need to synchronize anything, which is expected to be much much faster and reliable on your 10.000 disk system. It may also work over NFS.
The patch is complete, but not well tested. It just stopped working on it, after the regression test was successful and my box was able to reboot and remove the symlinks grabbed form the files :)
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
New patch with a fix for a bug introduced in udevinfo.
I am going to rerun my test with this patch applied. I will respond soon with the results.
Nice. It was just a bug with the -r option in udevinfo. If you don't use this db-query program, nothing has changed.
I do not use that program, but in any case the patch seems to have fixed the issues I was experiencing. I am able to add 1000 with no problem. I am going to try 10000 to make sure it is still successful. Thanks so much for your help!
Quoted 3 lines
Are you able to hotplug-remove the disks from the system? It would be nice to know, if the nodes and symlinks are all correctly removed if you use custom names, given by a rule for your thousands of disks setup.
So, the way scsi_debug is designed, I can remove devices by echo'ing a negative number into /sys/bus/pseudo/drivers/scsi_debug/add_host. Doing so seems to correctly remove the device nodes and symlinks. I am only now learning to use udev, so I will try the rule you mentioned and report those results once the 10000 disk test is finished.
Thanks again!
-Nish
-------------------------------------------------------
This SF.net email is sponsored by: IT Product Guide on ITManagersJournal Use IT products in your business? Tell us what you think of them. Give us Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
http://productguide.itmanagersjournal.com/guidepromo.tmpl
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel