Skip to contentSkip to conversation

implement a more efficient queue file format

PATCH In review Alan Jenkins 2 participants 3 messages May 21, 2009 – May 22, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 implement a more efficient queue file format

Directory lookups show up in profiling. The queue files are responsible for a large proportion of file-related system calls in udev coldplug.

Instead of creating a file for each event, append their details to a log file. The file is periodically rebuilt (garbage-collected) to prevent it from growing indefinitely.

This single queue file replaces both the queue directory and the uevent_seqnum file. On desktop systems the file tends not to grow beyond one page. So it should also save a small amount of memory in tmpfs.

Tests on a running EeePC indicate average savings of 5% *udevd* cpu time as measured by oprofile. __link_path_walk is reduced from 1.5% to 1.3%. It is not completely clear where the rest of the gains come from.

In tests running ~400 events, the queue file is rebuilt about 5 times.

---
 udev/Makefile.am                 |    1 +
 udev/lib/libudev-private.h       |   12 +-
 udev/lib/libudev-queue-export.c  |  447 ++++++++++++++++++++++++++++++++++++++
 udev/lib/libudev-queue-private.h |   51 +++++
 udev/lib/libudev-queue.c         |  311 ++++++++++++++++++--------
 udev/lib/libudev.h               |    2 +
 udev/udevadm-settle.c            |   22 +--
 udev/udevd.c                     |  154 +------------
 8 files changed, 744 insertions(+), 256 deletions(-)
 create mode 100644 udev/lib/libudev-queue-export.c
 create mode 100644 udev/lib/libudev-queue-private.h
udev/Makefile.am +1 -0
index fa8279d..6cd2f23 100644
@@ -30,6 +30,7 @@ common_files = \
 	lib/libudev-monitor.c \
 	lib/libudev-enumerate.c \
 	lib/libudev-queue.c \
+	lib/libudev-queue-export.c \
 	lib/libudev-ctrl.c
 
 if USE_SELINUX
udev/lib/libudev-private.h +7 -5
index 9ec5e1a..aa57b8b 100644
@@ -149,11 +149,13 @@ void udev_list_entry_set_flag(struct udev_list_entry *list_entry, int flag);
 	     entry != NULL; \
 	     entry = tmp, tmp = udev_list_entry_get_next(tmp))
 
-/* libudev-queue */
-int udev_queue_export_udev_seqnum(struct udev_queue *udev_queue, unsigned long long int seqnum);
-int udev_queue_export_device_queued(struct udev_queue *udev_queue, struct udev_device *udev_device);
-int udev_queue_export_device_finished(struct udev_queue *udev_queue, struct udev_device *udev_device);
-int udev_queue_export_device_failed(struct udev_queue *udev_queue, struct udev_device *udev_device);
+/* libudev-queue-export */
+struct udev_queue_export *udev_queue_export_new(struct udev *udev);
+void udev_queue_export_unref(struct udev_queue_export *udev_queue_export);
+void udev_queue_export_cleanup(struct udev_queue_export *udev_queue_export);
+int udev_queue_export_device_queued(struct udev_queue_export *udev_queue_export, struct udev_device *udev_device);
+int udev_queue_export_device_finished(struct udev_queue_export *udev_queue_export, struct udev_device *udev_device);
+int udev_queue_export_device_failed(struct udev_queue_export *udev_queue_export, struct udev_device *udev_device);
 
 /* libudev-utils */
 #define UTIL_PATH_SIZE				1024
udev/lib/libudev-queue-export.c new file +447 -0

447 lines not on this page · raw message

udev/lib/libudev-queue-private.h new file +51 -0

51 lines not on this page · raw message

udev/lib/libudev-queue.c +216 -95

406 lines not on this page · raw message

udev/lib/libudev.h +2 -0
index 9346eb4..a68540f 100644
@@ -115,6 +115,8 @@ unsigned long long int udev_queue_get_udev_seqnum(struct udev_queue *udev_queue)
 int udev_queue_get_udev_is_active(struct udev_queue *udev_queue);
 int udev_queue_get_queue_is_empty(struct udev_queue *udev_queue);
 int udev_queue_get_seqnum_is_finished(struct udev_queue *udev_queue, unsigned long long int seqnum);
+int udev_queue_get_seqnums_are_finished(struct udev_queue *udev_queue,
+				        unsigned long long int start, unsigned long long int end);
 struct udev_list_entry *udev_queue_get_queued_list_entry(struct udev_queue *udev_queue);
 struct udev_list_entry *udev_queue_get_failed_list_entry(struct udev_queue *udev_queue);
 #endif
udev/udevadm-settle.c +7 -15
index 52d9c0b..b416806 100644
@@ -173,24 +173,16 @@ int udevadm_settle(struct udev *udev, int argc, char *argv[])
 	}
 
 	while (!is_timeout) {
-		/* exit if queue is empty */
-		if (udev_queue_get_queue_is_empty(udev_queue))
-			break;
-
-		/* if asked for, wait for a specific sequence of events */
 		if (start > 0) {
-			unsigned long long seq;
-			int finished;
-
-			finished = 0;
-			for (seq = start; seq <= end; seq++) {
-				finished  = udev_queue_get_seqnum_is_finished(udev_queue, seq);
-				if (!finished)
-					break;
-			}
-			if (finished)
+			/* if asked for, wait for a specific sequence of events */
+			if (udev_queue_get_seqnums_are_finished(udev_queue, start, end) = 1)
+				break;
+		} else {
+			/* exit if queue is empty */
+			if (udev_queue_get_queue_is_empty(udev_queue))
 				break;
 		}
+
 		usleep(1000 * 1000 / LOOP_PER_SECOND);
 	}
 
udev/udevd.c +13 -141

216 lines not on this page · raw message

Signature · 1 line
1.6.0.4
#1 lore raw
Quoted 3 lines · Alan Jenkins

Instead of creating a file for each event, append their details to a log file.  The file is periodically rebuilt (garbage-collected) to prevent it from growing indefinitely.

Sounds cool. Boots fine here. :)

I've switched a few int to unsigned int, to silent some gcc warnings, merged the few entries in queue-private.h just into libudev-private.h, moved the file format comments to the C file, renamed the seqnums_are_finished() to seqnum_sequence_is_finished() and exported it, bumped the minor version of libudev, and comitted it.

Would be great, if you can check, that I didn't mess it up. :)

Thanks a lot,
Kay

#2 lore raw
In reply to Alan Jenkins
Quoted 15 lines · Kay Sievers
Quoted 4 lines · Alan Jenkins

Instead of creating a file for each event, append their details to a log file. The file is periodically rebuilt (garbage-collected) to prevent it from growing indefinitely.

Sounds cool. Boots fine here. :)

I've switched a few int to unsigned int, to silent some gcc warnings, merged the few entries in queue-private.h just into libudev-private.h, moved the file format comments to the C file, renamed the seqnums_are_finished() to seqnum_sequence_is_finished() and exported it, bumped the minor version of libudev, and comitted it.

Quoted 5 lines

Would be great, if you can check, that I didn't mess it up. :)

Thanks a lot,
Kay

Wow, thanks! I checked the delta. I don't see any problems in these changes.

I only see one warning myself

lib/libudev-queue-export.c:143: warning: comparison between signed and unsigned

142 n = seqnum - udev_queue_export->seqnum_max;
143 if (n >= devpaths->devpaths_size)
144 goto read_error;

Arguably it _might_ be better addressed in this one place, by casting devpaths_size to unsigned. It seems to be a matter of personal taste though, and I'm not complaining.

I have an unverifiable memory of reading "unsigned int i is evil" on LKML. Half the internet says unsigned has relatively special semantics (intended for e.g. bit manipulation), you should avoid it unless that's what you actually need, and size_t should never have been defined as unsigned. Of course, the other half advocate using unsigned liberally to exclude negative values when passing array indexes around :-).

Alan