Pseudo performance changes...
cover Pseudo performance changes...
Unlike most of my submissions, this isn't patches against oe-core; rather, it's patches against pseudo, and if I can get some confirmation that they do what I think they do, and some review, I'm planning to make this into pseudo 1.5, and send a patch "soonish" to merge that into oe-core.
What this does: Fix a number of build performance issues. By far the largest change is actually not so much a problem with pseudo as a problem that pseudo can solve by brute force. Packaging systems (at least RPM and SMART) do a lot of fsync() and fdatasync() calls. That usually implies flushing EVERYTHING that's been written, not just one specific file. And that, in turn, results in a severe performance hit.
So, for instance, on one of my test workstations, this moves a do_rootfs with about 1200 RPMs from about 22 minutes to about 4.5. Yeah.
The other changes aren't as dramatic for that case, but have very significant performance impact for at least some workloads. The first is switching to using an in-memory database for the files database, dumping it to disk only when the pseudo daemon is idle or shutting down. This doesn't produce huge benefits in all cases, but for workloads with a lot of parallelism, it can produce a very noticeable reduction in how much pseudo slows things down.
The second is a fairly major protocol change. In short, with this patch, pseudo clients only wait for a server response when they need information from the server in order to continue. That's OP_FSTAT, OP_STAT, OP_MAY_UNLINK, and OP_MKNOD. Everything else just silently assumes that it probably succeeded.
How much does this matter? Between the protocol change and the memory DB, a trivial unpack of a tarball (lots of writes to the database, very few reads) can be about 4x faster. Removing stuff isn't much faster, but it might be a bit faster.
This is most noticeable, by far, when running more than one build, or when running builds while doing other things. It has a much smaller effect on builds with no shared state (compile time still dominates that), but even there I'm seeing decreases from ~83 minutes to ~64 from just the fsync and memory changes. Still waiting on my real test case (multiple simultaneous builds which need compiles) completing.
Peter Seebach (3):
Use in-memory database for files
allow pseudo to force asynchronous behavior If you don't want the answer, don't ask the question.
ChangeLog.txt | 10 ++++ Makefile.in | 6 +- configure | 51 ++++++++++++++++ enums/msg_type.in | 1 + enums/op.in | 46 +++++++------- enums/query_type.in | 18 +++--- guts/COPYRIGHT | 2 +- maketables | 41 +++++++------ makewrappers | 13 +++- ports/darwin/guts/open.c | 5 +- ports/linux/guts/openat.c | 21 ++++++- ports/unix/guts/fchmod.c | 16 ++--- ports/unix/guts/fchmodat.c | 31 ++++++---- ports/unix/guts/fchown.c | 16 ++--- ports/unix/guts/fchownat.c | 16 ++--- ports/unix/guts/fdatasync.c | 16 +++++ ports/unix/guts/fsync.c | 16 +++++ ports/unix/guts/mknodat.c | 4 +- ports/unix/guts/msync.c | 16 +++++ ports/unix/guts/sync.c | 16 +++++ ports/unix/guts/sync_file_range.c | 13 ++++ ports/unix/guts/syncfs.c | 13 ++++ ports/unix/wrapfuncs.in | 9 +++ pseudo.c | 3 +- pseudo_client.c | 22 ++++--- pseudo_db.c | 119 ++++++++++++++++++++++++++++++++++++- pseudo_db.h | 1 + pseudo_server.c | 32 ++++++---- templates/wrapfuncs.c | 2 + 29 files changed, 453 insertions(+), 122 deletions(-) create mode 100644 ports/unix/guts/fdatasync.c create mode 100644 ports/unix/guts/fsync.c create mode 100644 ports/unix/guts/msync.c create mode 100644 ports/unix/guts/sync.c create mode 100644 ports/unix/guts/sync_file_range.c create mode 100644 ports/unix/guts/syncfs.c
Signature · 1 line
1.7.9.5
PATCH 1/3 Use in-memory database for files
It turns out that file databases don't get very large, and that sqlite3 can be quite fast with an in-memory database. It also turns out that dumping the database to disk on exit (or during idle times) is pretty cheap compared to constant updates.
So: We add "--enable-memory-db", which defaults to on if you have sqlite 3.7 or later, and off for 3.6 (because 3.6 has horrible performance with in-memory db on some hosts we tried).
--- ChangeLog.txt | 5 +++ Makefile.in | 5 ++- configure | 32 +++++++++++++++ pseudo_db.c | 119 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- pseudo_db.h | 1 + pseudo_server.c | 2 + 6 files changed, 159 insertions(+), 5 deletions(-)
ChangeLog.txt +5 -0
@@ -1,3 +1,8 @@ +2013-02-15: + * (seebs) Add support for in-memory DB. This, plus upcoming + fsync-related changes, are expected to be big enough to justify + calling this 1.5. + 2013-02-13: * (seebs) calling link while chrooted could in some cases result in the root path not being prepended at all. One more try!
Makefile.in +3 -2
@@ -23,11 +23,12 @@ LIBDIR=@LIBDIR@ SUFFIX=@SUFFIX@ SQLITE=@SQLITE@ SQLITE_LIB=@SQLITE_LIB@ +SQLITE_MEMORY=@SQLITE_MEMORY@ BITS=@BITS@ ARCH_FLAGS=@ARCH_FLAGS@ MARK64=@MARK64@ RPATH=@RPATH@ -VERSION=1.4.5 +VERSION=1.5 LIB=@LIB@ BIN=bin
@@ -37,7 +38,7 @@ LOCALSTATEDIR=$(PREFIX)/$(LOCALSTATE) CFLAGS_BASE=-pipe -std=gnu99 -Wall -W -Wextra CFLAGS_CODE=-fPIC -D_LARGEFILE64_SOURCE -D_ATFILE_SOURCE $(ARCH_FLAGS) -CFLAGS_DEFS=-DPSEUDO_PREFIX='"$(PREFIX)"' -DPSEUDO_SUFFIX='"$(SUFFIX)"' -DPSEUDO_BINDIR='"$(BIN)"' -DPSEUDO_LIBDIR='"$(LIB)"' -DPSEUDO_LOCALSTATEDIR='"$(LOCALSTATE)"' -DPSEUDO_VERSION='"$(VERSION)"' +CFLAGS_DEFS=-DPSEUDO_PREFIX='"$(PREFIX)"' -DPSEUDO_SUFFIX='"$(SUFFIX)"' -DPSEUDO_BINDIR='"$(BIN)"' -DPSEUDO_LIBDIR='"$(LIB)"' -DPSEUDO_LOCALSTATEDIR='"$(LOCALSTATE)"' -DPSEUDO_VERSION='"$(VERSION)"' $(SQLITE_MEMORY) CFLAGS_DEBUG=-O2 -g CFLAGS_SQL=-L$(SQLITE)/$(SQLITE_LIB) -I$(SQLITE)/include $(RPATH) CFLAGS_PSEUDO=$(CFLAGS_BASE) $(CFLAGS_CODE) $(CFLAGS_DEFS) \
configure +32 -0
@@ -25,6 +25,7 @@ opt_arch=x86 opt_bits= opt_sqlite=/usr opt_rpath= +opt_memory= compile_x86_32=-m32 compile_x86_64=-m64
@@ -35,6 +36,7 @@ usage() echo >&2 " configure --prefix=..." echo >&2 " [--libdir=...]" echo >&2 " [--suffix=...]" + echo >&2 " [--enable-memory-db]" echo >&2 " [--with-sqlite=...]" echo >&2 " [--with-sqlite-lib=...]" echo >&2 " [--enable-static-sqlite]"
@@ -69,6 +71,12 @@ do sqlite_ldarg='$(SQLITE)/$(SQLITE_LIB)/libsqlite3.a' use_maybe_rpath=false ;; + --enable-memory-db=no) + opt_memory=false + ;; + --enable-memory-db=yes | --enable-memory-db) + opt_memory=true + ;; --with-sqlite=*) opt_sqlite=${arg#--with-sqlite=} # assign new value if unset
@@ -170,6 +178,29 @@ if [ "${SQLITE3_VERSION}" -lt "03006000" ]; then exit 1 fi +if [ -z "$opt_memory" ]; then + if [ "${SQLITE3_VERSION}" -lt "03007000" ]; then + echo "Disabling in-memory database by default (sqlite too old)." + opt_memory=false + else + echo "Enabling in-memory database by default." + opt_memory=true + fi +fi + +if $opt_memory; then + if [ "${SQLITE3_VERSION}" -lt "03007000" ]; then + cat >&2 <<EOF +WARNING: sqlite prior to 3.7 has been known to perform exceedingly poorly +with the in-memory database option. You asked for it, you get it, but if +you get horrible performance, try turning it off. +EOF + fi + SQLITE_MEMORY="-DUSE_MEMORY_DB" +else + SQLITE_MEMORY="" +fi + sed -e ' s,@PREFIX@,'"$opt_prefix"',g s,@LIBDIR@,'"$opt_libdir"',g
@@ -179,6 +210,7 @@ sed -e ' s,@ARCH_FLAGS@,'"$arch_flags"',g s,@SQLITE_LDARG@,'"$sqlite_ldarg"',g s,@SQLITE_LIB@,'"$opt_sqlite_lib"',g + s,@SQLITE_MEMORY@,'"$SQLITE_MEMORY"',g s!@RPATH@!'"$opt_rpath"'!g s,@MARK64@,'"$opt_mark64"',g s,@ARCH@,'"$opt_arch"',g
pseudo_db.c +116 -3
263 lines not on this page · raw message
pseudo_db.h +1 -0
@@ -37,6 +37,7 @@ typedef struct {char *program; } log_entry; +extern int pdb_maybe_backup (void );extern int pdb_cancel_unlink_file (pseudo_msg_t *msg);extern int pdb_did_unlink_file (char *path,int deleting);extern int pdb_did_unlink_files (int deleting);
pseudo_server.c +2 -0
@@ -416,6 +416,8 @@ pseudo_server_loop(void) {*/ if (active_clients ==1 ) { loop_timeout -=LOOP_DELAY ; +/* maybe flush database to disk */ +pdb_maybe_backup ();if (loop_timeout <=0 ) {pseudo_debug (1 ,"no more clients, got bored.\n" ); die_peacefully =1 ;
Signature · 1 line
1.7.9.5
PATCH 2/3 allow pseudo to force asynchronous behavior
In reply to Peter SeebachThe openembedded build, at least with RPM or SMART, is heavily affected by the cost of calling fsync or fdatasync on package databases all the time. Gosh, wouldn't it be nice if we could suppress that without making dozens of highly intrusive and risky changes into RPM, various database packages, and so on?
Yes, yes it would. If only there were a program which could intercept system calls and change their behavior!
Enter --enable-force-async. There are now wrappers for fsync, fdatasync, and a few related functions. If --enable-force-async is set, these wrappers instantly return 0, even if PSEUDO_DISABLED is set. And with any luck, bitbake will now perform a bit better.
Credit for this insight goes to Richard Purdie. I've reimplemented this to add the configure option, and make the fsync suppression work even when PSEUDO_DISABLED is set.
--- ChangeLog.txt | 5 +++++ Makefile.in | 3 ++- configure | 19 +++++++++++++++++++ guts/COPYRIGHT | 2 +- makewrappers | 13 ++++++++++++- ports/darwin/guts/open.c | 5 ++++- ports/linux/guts/openat.c | 21 ++++++++++++++++++++- ports/unix/guts/fdatasync.c | 16 ++++++++++++++++ ports/unix/guts/fsync.c | 16 ++++++++++++++++ ports/unix/guts/msync.c | 16 ++++++++++++++++ ports/unix/guts/sync.c | 16 ++++++++++++++++ ports/unix/guts/sync_file_range.c | 13 +++++++++++++ ports/unix/guts/syncfs.c | 13 +++++++++++++ ports/unix/wrapfuncs.in | 9 +++++++++ templates/wrapfuncs.c | 2 ++ 15 files changed, 164 insertions(+), 5 deletions(-) create mode 100644 ports/unix/guts/fdatasync.c create mode 100644 ports/unix/guts/fsync.c create mode 100644 ports/unix/guts/msync.c create mode 100644 ports/unix/guts/sync.c create mode 100644 ports/unix/guts/sync_file_range.c create mode 100644 ports/unix/guts/syncfs.c
ChangeLog.txt +5 -0
@@ -1,3 +1,8 @@ +2013-02-16: + * (seebs) Add wrapper setting which automatically returns a fixed + value from some wrappers. Add fixed-value wrappers for all the + *sync() functions. These are all contingent on --enable-force-async. + 2013-02-15: * (seebs) Add support for in-memory DB. This, plus upcoming fsync-related changes, are expected to be big enough to justify
Makefile.in +2 -1
@@ -24,6 +24,7 @@ SUFFIX=@SUFFIX@ SQLITE=@SQLITE@ SQLITE_LIB=@SQLITE_LIB@ SQLITE_MEMORY=@SQLITE_MEMORY@ +FORCE_ASYNC=@FORCE_ASYNC@ BITS=@BITS@ ARCH_FLAGS=@ARCH_FLAGS@ MARK64=@MARK64@
@@ -38,7 +39,7 @@ LOCALSTATEDIR=$(PREFIX)/$(LOCALSTATE) CFLAGS_BASE=-pipe -std=gnu99 -Wall -W -Wextra CFLAGS_CODE=-fPIC -D_LARGEFILE64_SOURCE -D_ATFILE_SOURCE $(ARCH_FLAGS) -CFLAGS_DEFS=-DPSEUDO_PREFIX='"$(PREFIX)"' -DPSEUDO_SUFFIX='"$(SUFFIX)"' -DPSEUDO_BINDIR='"$(BIN)"' -DPSEUDO_LIBDIR='"$(LIB)"' -DPSEUDO_LOCALSTATEDIR='"$(LOCALSTATE)"' -DPSEUDO_VERSION='"$(VERSION)"' $(SQLITE_MEMORY) +CFLAGS_DEFS=-DPSEUDO_PREFIX='"$(PREFIX)"' -DPSEUDO_SUFFIX='"$(SUFFIX)"' -DPSEUDO_BINDIR='"$(BIN)"' -DPSEUDO_LIBDIR='"$(LIB)"' -DPSEUDO_LOCALSTATEDIR='"$(LOCALSTATE)"' -DPSEUDO_VERSION='"$(VERSION)"' $(SQLITE_MEMORY) $(FORCE_ASYNC) CFLAGS_DEBUG=-O2 -g CFLAGS_SQL=-L$(SQLITE)/$(SQLITE_LIB) -I$(SQLITE)/include $(RPATH) CFLAGS_PSEUDO=$(CFLAGS_BASE) $(CFLAGS_CODE) $(CFLAGS_DEFS) \
configure +19 -0
@@ -26,6 +26,7 @@ opt_bits= opt_sqlite=/usr opt_rpath= opt_memory= +opt_async= compile_x86_32=-m32 compile_x86_64=-m64
@@ -37,6 +38,7 @@ usage() echo >&2 " [--libdir=...]" echo >&2 " [--suffix=...]" echo >&2 " [--enable-memory-db]" + echo >&2 " [--enable-force-async]" echo >&2 " [--with-sqlite=...]" echo >&2 " [--with-sqlite-lib=...]" echo >&2 " [--enable-static-sqlite]"
@@ -71,6 +73,12 @@ do sqlite_ldarg='$(SQLITE)/$(SQLITE_LIB)/libsqlite3.a' use_maybe_rpath=false ;; + --enable-force-async=no | --disable-force-async) + opt_async=false + ;; + --enable-force-async=yes | --enable-force-async) + opt_async=true + ;; --enable-memory-db=no) opt_memory=false ;;
@@ -178,6 +186,16 @@ if [ "${SQLITE3_VERSION}" -lt "03006000" ]; then exit 1 fi +if [ -z "$opt_async" ]; then + opt_async=false +fi + +if $opt_async; then + FORCE_ASYNC="-DPSEUDO_FORCE_ASYNC" +else + FORCE_ASYNC="" +fi + if [ -z "$opt_memory" ]; then if [ "${SQLITE3_VERSION}" -lt "03007000" ]; then echo "Disabling in-memory database by default (sqlite too old)."
@@ -211,6 +229,7 @@ sed -e ' s,@SQLITE_LDARG@,'"$sqlite_ldarg"',g s,@SQLITE_LIB@,'"$opt_sqlite_lib"',g s,@SQLITE_MEMORY@,'"$SQLITE_MEMORY"',g + s,@FORCE_ASYNC@,'"$FORCE_ASYNC"',g s!@RPATH@!'"$opt_rpath"'!g s,@MARK64@,'"$opt_mark64"',g s,@ARCH@,'"$opt_arch"',g
guts/COPYRIGHT +1 -1
@@ -1,5 +1,5 @@ /* - * Copyright (c) 2008-2010 Wind River Systems, Inc. + * Copyright (c) 2008-2013 Wind River Systems, Inc. * * This program is free software; you can redistribute it and/or modify * it under the terms of the Lesser GNU General Public License version 2.1 as
makewrappers +12 -1
@@ -1,6 +1,6 @@ #!/usr/bin/env python # -# Copyright (c) 2008-2011 Wind River Systems, Inc. +# Copyright (c) 2008-2011,2013 Wind River Systems, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the Lesser GNU General Public License version 2.1 as
@@ -219,6 +219,7 @@ class Function: self.paths_to_munge = [] self.specific_dirfds = {} self.hand_wrapped = None + self.async_skip = None # used for the copyright date when creating stub functions self.date = datetime.date.today().year
@@ -300,6 +301,16 @@ class Function: else: return "" + def maybe_async_skip(self): + if self.async_skip: + return """/* This function is not called if pseudo is configured --enable-force-async */ +#ifdef PSEUDO_FORCE_ASYNC + return %s; +#endif +""" % self.async_skip + else: + return "" + def comment(self): """declare self (in a comment)""" return self.decl(comment = True)
ports/darwin/guts/open.c +4 -1
@@ -1,5 +1,5 @@/* -* Copyright (c) 2011, 2012 Wind River Systems; see +* Copyright (c) 2011-2013 Wind River Systems; see * guts/COPYRIGHT for information. * * int open(const char *path, int flags, ... { int mode })
@@ -9,6 +9,9 @@struct stat buf = { };int existed =1 ;int save_errno; +#ifdef PSEUDO_FORCE_ASYNCH + flags &= ~O_SYNC ; +#endif /* if a creation has been requested, check whether file exists */ if (flags &O_CREAT ) {
ports/linux/guts/openat.c +20 -1
@@ -1,5 +1,5 @@/* -* Copyright (c) 2008-2010 Wind River Systems; see +* Copyright (c) 2008-2010, 2013 Wind River Systems; see * guts/COPYRIGHT for information. * * static int
@@ -16,6 +16,25 @@return -1 ; }#endif + +#ifdef PSEUDO_FORCE_ASYNCH +/* Yes, I'm aware that every Linux system I've seen has +* DSYNC and RSYNC being the same value as SYNC. +*/ + + flags &= ~(O_SYNC +#ifdef O_DIRECT + |O_DIRECT +#endif +#ifdef O_DSYNC + |O_DSYNC +#endif +#ifdef O_RSYNC + |O_RSYNC +#endif + ); +#endif +/* if a creation has been requested, check whether file exists */ if (flags &O_CREAT ) { save_errno = errno;
ports/unix/guts/fdatasync.c new file +16 -0
@@ -0,0 +1,16 @@ +/* +* Copyright (c) 2013 Wind River Systems; see +* guts/COPYRIGHT for information. +* +* int fdatasync(int fd) +* int rc = -1; +*/ + +/* note: wrapper will never call this if PSEUDO_FORCE_ASYNC +* is defined. +*/ + rc =real_fdatasync (fd); + +/* return rc; +* } +*/
ports/unix/guts/fsync.c new file +16 -0
@@ -0,0 +1,16 @@ +/* +* Copyright (c) 2013 Wind River Systems; see +* guts/COPYRIGHT for information. +* +* int fsync(int fd) +* int rc = -1; +*/ + +/* note: wrapper will never call this if PSEUDO_FORCE_ASYNC +* is defined. +*/ + rc =real_fsync (fd); + +/* return rc; +* } +*/
ports/unix/guts/msync.c new file +16 -0
@@ -0,0 +1,16 @@ +/* +* Copyright (c) 2013 Wind River Systems; see +* guts/COPYRIGHT for information. +* +* int msync(void *addr, size_t length, int flags) +* int rc = -1; +*/ + +/* note: wrapper will never call this if PSEUDO_FORCE_ASYNC +* is defined. +*/ + rc =real_msync (addr, length, flags); + +/* return rc; +* } +*/
ports/unix/guts/sync.c new file +16 -0
@@ -0,0 +1,16 @@ +/* +* Copyright (c) 2013 Wind River Systems; see +* guts/COPYRIGHT for information. +* +* void sync(void) +* +*/ + +/* note: wrapper will never call this if PSEUDO_FORCE_ASYNC +* is defined. +*/ + (void )real_sync (); + +/* return; +* } +*/
ports/unix/guts/sync_file_range.c new file +13 -0
@@ -0,0 +1,13 @@ +/* +* Copyright (c) 2013 Wind River Systems; see +* guts/COPYRIGHT for information. +* +* int sync_file_range(int fd, off64_t offset, off64_t nbytes, unsigned int flags) +* int rc = -1; +*/ + + rc =real_sync_file_range (fd, offset, nbytes, flags); + +/* return rc; +* } +*/
ports/unix/guts/syncfs.c new file +13 -0
@@ -0,0 +1,13 @@ +/* +* Copyright (c) 2013 Wind River Systems; see +* guts/COPYRIGHT for information. +* +* int syncfs(int fd) +* int rc = -1; +*/ + + rc =real_syncfs (fd); + +/* return rc; +* } +*/
ports/unix/wrapfuncs.in +9 -0
@@ -58,3 +58,12 @@ ssize_t readlink(const char *path, char *buf, size_t bufsiz); /* flags=AT_SYMLIN ssize_t readlinkat(int dirfd, const char *path, char *buf, size_t bufsiz); /* flags=AT_SYMLINK_NOFOLLOW */ int system(const char *command); FILE *popen(const char *command, const char *mode); /* hand_wrapped=1 */ +# Based on experiments by Richard Purdie: Allow pseudo to eliminate +# sync-type operations globally, mostly relevant for performance reasons +# during filesystem assembly. +int fsync(int fd); /* async_skip=0 */ +int fdatasync(int fd); /* async_skip=0 */ +void sync(void); /* async_skip=0 */ +int syncfs(int fd); /* async_skip=0 */ +int sync_file_range(int fd, off64_t offset, off64_t nbytes, unsigned int flags); /* async_skip=0 */ +int msync(void *addr, size_t length, int flags); /* async_skip=0 */
templates/wrapfuncs.c +2 -0
@@ -23,6 +23,8 @@ ${name}(${decl_args}) { ${variadic_decl} ${rc_decl} +${maybe_async_skip} + if (!pseudo_check_wrappers() || !real_$name) { /* rc was initialized to the "failure" value */ pseudo_enosys("${name}");
Signature · 1 line
1.7.9.5
PATCH 3/3 If you don't want the answer, don't ask the question.
In reply to Peter SeebachMost pseudo operations don't actually USE the server's response. So why wait for a response?
This patch introduces a new message type, PSEUDO_MSG_FASTOP. It also tags pseudo operation types with whether or not they need to give a response. This requires updates to maketables to allow non-string types for additional columns, and the addition of some quotes to the SQL query enums/query_type.in table.
A few routines are altered to change their behavior and whether or not they perform a stat operation. The only operations that do wait are OP_FSTAT and OP_STAT, OP_MKNOD, and OP_MAY_UNLINK. Rationale:
You can't query the server for replacement information and not wait for it. Makes no sense.
There's extra checking in mknod, because we really do want to fail out if we couldn't do that -- that implies that we haven't created a thing that will look like a node.
The result from OP_MAY_UNLINK is checked because it's used to determine whether we need to send a DID_UNLINK or CANCEL_UNLINK. It might be cheaper to send two messages without waiting than to send one, wait, and maybe send another, but I don't want to send invalid messages.
This is highly experimental.
--- enums/msg_type.in | 1 + enums/op.in | 46 ++++++++++++++++++++++---------------------- enums/query_type.in | 18 ++++++++--------- maketables | 41 +++++++++++++++++++++++---------------- ports/unix/guts/fchmod.c | 16 +++++---------- ports/unix/guts/fchmodat.c | 31 ++++++++++++++++------------- ports/unix/guts/fchown.c | 16 ++++++--------- ports/unix/guts/fchownat.c | 16 ++++++--------- ports/unix/guts/mknodat.c | 4 +++- ports/unix/wrapfuncs.in | 2 +- pseudo.c | 3 ++- pseudo_client.c | 22 +++++++++++++-------- pseudo_server.c | 30 +++++++++++++++++++---------- 13 files changed, 132 insertions(+), 114 deletions(-)
enums/msg_type.in +1 -0
@@ -4,3 +4,4 @@ shutdown op ack nak +fastop
enums/op.in +23 -23
46 lines not on this page · raw message
enums/query_type.in +9 -9
@@ -1,9 +1,9 @@ -query_type: PSQT; sql = LITTLE BOBBY TABLES -exact, = -less, < -greater, > -bitand, & -notequal, != -like, LIKE -notlike, NOT LIKE -sqlpat, LIKE +query_type: PSQT; const char * sql = "LITTLE BOBBY TABLES" +exact, "=" +less, "<" +greater, ">" +bitand, "&" +notequal, "!=" +like, "LIKE" +notlike, "NOT LIKE" +sqlpat, "LIKE"
maketables +24 -17
98 lines not on this page · raw message
ports/unix/guts/fchmod.c +5 -11
@@ -1,12 +1,11 @@/* -* Copyright (c) 2008-2010, 2012 Wind River Systems; see +* Copyright (c) 2008-2010, 2012, 2013 Wind River Systems; see * guts/COPYRIGHT for information. * * static int * wrap_fchmod(int fd, mode_t mode) { * int rc = -1; */ -pseudo_msg_t *msg;PSEUDO_STATBUF buf;int save_errno = errno;
@@ -15,16 +14,11 @@return -1 ; } buf.st_mode = (buf.st_mode & ~07777 ) | (mode &07777 ); - msg =pseudo_client_op (OP_FCHMOD ,0 , fd,-1 ,0 , &buf); +pseudo_client_op (OP_FCHMOD ,0 , fd,-1 ,0 , &buf);real_fchmod (fd,PSEUDO_FS_MODE (mode,S_ISDIR (buf.st_mode))); -if (msg && msg->result !=RESULT_SUCCEED ) { - errno =EPERM ; - rc =-1 ; - }else { -/* just pretend we worked */ - errno = save_errno; - rc =0 ; - } +/* just pretend we worked */ + errno = save_errno; + rc =0 ;/* return rc; * }
ports/unix/guts/fchmodat.c +18 -13
@@ -1,14 +1,13 @@/* -* Copyright (c) 2008-2010, 2012 Wind River Systems; see +* Copyright (c) 2008-2010, 2012, 2013 Wind River Systems; see * guts/COPYRIGHT for information. * * static int * wrap_fchmodat(int dirfd, const char *path, mode_t mode, int flags) { * int rc = -1; */ -pseudo_msg_t *msg;PSEUDO_STATBUF buf; -int save_errno; +int save_errno = errno;#ifdef PSEUDO_NO_REAL_AT_FUNCTIONS if (dirfd !=AT_FDCWD ) {
@@ -33,8 +32,13 @@ } save_errno = errno; +#if 0 +pseudo_msg_t *msg;/* purely for debugging purposes: check whether file -* is already in database. +* is already in database. We don't need the resulting +* information for anything. This is currently ifdefed +* out because it's only useful when trying to track where +* files are coming from. */ msg =pseudo_client_op (OP_STAT ,0 ,-1 ,-1 , path, &buf);if (!msg || msg->result !=RESULT_SUCCEED ) {
@@ -42,6 +46,7 @@ mode, dirfd, path, (unsigned long long) buf.st_ino); } +#endif /* user bits added so "root" can always access files. */ #ifdef PSEUDO_NO_REAL_AT_FUNCTIONS
@@ -53,18 +58,18 @@#endif /* we ignore a failure from underlying fchmod, because pseudo * may believe you are permitted to change modes that the filesystem -* doesn't. +* doesn't. Note that we also don't need to know whether the +* file might be a (pseudo) block device or some such; pseudo +* will only modify permission bits based on an OP_CHMOD, and does +* not care about device/file type mismatches, only directory/file +* or symlink/file. */ buf.st_mode = (buf.st_mode & ~07777 ) | (mode &07777 ); - msg =pseudo_client_op (OP_CHMOD ,0 ,-1 , dirfd, path, &buf); -if (msg && msg->result !=RESULT_SUCCEED ) { - errno =EPERM ; - rc =-1 ; - }else { -/* if server is down, just pretend we worked */ - rc =0 ; - } +pseudo_client_op (OP_CHMOD ,0 ,-1 , dirfd, path, &buf); +/* don't change errno from what it was originally */ + errno = save_errno; + rc =0 ;/* return rc; * }
ports/unix/guts/fchown.c +6 -10
@@ -1,5 +1,5 @@/* -* Copyright (c) 2008-2010, 2012 Wind River Systems; see +* Copyright (c) 2008-2010, 2012, 2013 Wind River Systems; see * guts/COPYRIGHT for information. * * static int
@@ -8,7 +8,7 @@*/ pseudo_msg_t *msg;PSEUDO_STATBUF buf; -int save_errno; +int save_errno = errno;if (base_fstat (fd, &buf) ==-1 ) { save_errno = errno;
@@ -41,14 +41,10 @@ }pseudo_debug (2 ,"fchown, fd %d: %d:%d -> %d:%d\n" , fd, owner, group, buf.st_uid, buf.st_gid); - msg =pseudo_client_op (OP_FCHOWN ,0 , fd,-1 ,0 , &buf); -if (msg && msg->result !=RESULT_SUCCEED ) { - errno =EPERM ; - rc =-1 ; - }else { -/* just pretend we worked */ - rc =0 ; - } +pseudo_client_op (OP_FCHOWN ,0 , fd,-1 ,0 , &buf); +/* pretend we worked, errno should be unchanged */ + errno = save_errno; + rc =0 ;/* return rc; * }
ports/unix/guts/fchownat.c +6 -10
@@ -1,5 +1,5 @@/* -* Copyright (c) 2008-2010, 2012 Wind River Systems; see +* Copyright (c) 2008-2010, 2012, 2013 Wind River Systems; see * guts/COPYRIGHT for information. * * static int
@@ -8,7 +8,7 @@*/ pseudo_msg_t *msg;PSEUDO_STATBUF buf; -int save_errno; +int save_errno = errno;int doing_link =0 ;#ifdef PSEUDO_NO_REAL_AT_FUNCTIONS
@@ -53,14 +53,10 @@if (group != (gid_t) -1 ) { buf.st_gid = group; } - msg =pseudo_client_op (OP_CHOWN ,0 ,-1 , dirfd, path, &buf); -if (msg && msg->result !=RESULT_SUCCEED ) { - errno =EPERM ; - rc =-1 ; - }else { -/* just pretend we worked */ - rc =0 ; - } +pseudo_client_op (OP_CHOWN ,0 ,-1 , dirfd, path, &buf); +/* just pretend we worked */ + errno = save_errno; + rc =0 ;/* return rc; * }
ports/unix/guts/mknodat.c +3 -1
@@ -8,6 +8,7 @@pseudo_msg_t *msg;PSEUDO_STATBUF buf; +int save_errno = errno;#ifdef PSEUDO_NO_REAL_AT_FUNCTIONS if (dirfd !=AT_FDCWD ) {
@@ -50,10 +51,11 @@ rc =-1 ; }else {/* just pretend we worked */ + errno = save_errno; rc =0 ; }if (rc ==-1 ) { -int save_errno = errno; + save_errno = errno;#ifdef PSEUDO_NO_REAL_AT_FUNCTIONS real_unlink (path);#else
ports/unix/wrapfuncs.in +1 -1
@@ -63,7 +63,7 @@ FILE *popen(const char *command, const char *mode); /* hand_wrapped=1 */ # during filesystem assembly. int fsync(int fd); /* async_skip=0 */ int fdatasync(int fd); /* async_skip=0 */ -void sync(void); /* async_skip=0 */ +void sync(void); /* async_skip= */ int syncfs(int fd); /* async_skip=0 */ int sync_file_range(int fd, off64_t offset, off64_t nbytes, unsigned int flags); /* async_skip=0 */ int msync(void *addr, size_t length, int flags); /* async_skip=0 */
pseudo.c +2 -1
@@ -1,7 +1,7 @@/* * pseudo.c, main pseudo utility program * -* Copyright (c) 2008-2010 Wind River Systems, Inc. +* Copyright (c) 2008-2013 Wind River Systems, Inc. * * This program is free software; you can redistribute it and/or modify * it under the terms of the Lesser GNU General Public License version 2.1 as
@@ -933,6 +933,7 @@ pseudo_server_response(pseudo_msg_t *msg, const char *program, const char *tag)return 0 ;break ;case PSEUDO_MSG_OP : +case PSEUDO_MSG_FASTOP :return pseudo_op (msg, program, tag);break ;case PSEUDO_MSG_ACK :/* FALLTHROUGH */
pseudo_client.c +14 -8
@@ -876,14 +876,18 @@ pseudo_client_request(pseudo_msg_t *msg, size_t len, const char *path) { } }while (rc !=0 );pseudo_debug (5 ,"sent!\n" ); - response =pseudo_msg_receive (connect_fd); -if (!response) { - ++tries; -if (tries >3 ) { -pseudo_debug (1 ,"can't get responses.\n" ); -return 0 ; - } - } +if (msg->type !=PSEUDO_MSG_FASTOP ) { + response =pseudo_msg_receive (connect_fd); +if (!response) { + ++tries; +if (tries >3 ) { +pseudo_debug (1 ,"can't get responses.\n" ); +return 0 ; + } + } + }else { +return 0 ; + } }while (response ==0 );if (response->type !=PSEUDO_MSG_ACK ) {pseudo_debug (2 ,"got non-ack response %d\n" , response->type);
@@ -1225,6 +1229,8 @@ pseudo_client_op(pseudo_op_t op, int access, int fd, int dirfd, const char *path }if (do_request) {struct timeval tv1, tv2; +if (!pseudo_op_wait (msg.op)) + msg.type =PSEUDO_MSG_FASTOP ;pseudo_debug (4 ,"sending request [ino %llu]\n" , (unsigned long long ) msg.ino);gettimeofday (&tv1,NULL );if (pseudo_local_only) {
pseudo_server.c +20 -10
@@ -1,7 +1,7 @@/* * pseudo_server.c, pseudo's server-side logic and message handling * -* Copyright (c) 2008-2010 Wind River Systems, Inc. +* Copyright (c) 2008-2010, 2013 Wind River Systems, Inc. * * This program is free software; you can redistribute it and/or modify * it under the terms of the Lesser GNU General Public License version 2.1 as
@@ -76,7 +76,7 @@ quit_now(int signal) { die_forcefully =1 ; } -static int messages =0 ; +static int messages =0 , responses =0 ;static struct timeval message_time = { .tv_sec =0 };static void pseudo_server_loop (void );
@@ -268,6 +268,7 @@ serve_client(int i) { in =pseudo_msg_receive (clients[i].fd);if (in) {char *response_path =0 ; +int send_response =1 ;pseudo_debug (4 ,"got a message (%d): %s\n" , in->type, (in->pathlen ? in->path :"<no path>" ));/* handle incoming ping */ if (in->type ==PSEUDO_MSG_PING && !clients[i].pid) {
@@ -303,6 +304,8 @@ serve_client(int i) {* pseudo_server_response. */ if (in->type !=PSEUDO_MSG_SHUTDOWN ) { +if (in->type ==PSEUDO_MSG_FASTOP ) + send_response =0 ;if (pseudo_server_response (in, clients[i].program, clients[i].tag)) { in->type =PSEUDO_MSG_NAK ; }else {
@@ -343,10 +346,14 @@ serve_client(int i) { die_peacefully =1 ; } } -if ((rc =pseudo_msg_send (clients[i].fd, in,-1 , response_path)) !=0 ) -pseudo_debug (1 ,"failed to send response to client %d [%d]: %d (%s)\n" , - i, (int ) clients[i].pid, rc,strerror (errno)); - rc = in->op; +if (send_response) { +if ((rc =pseudo_msg_send (clients[i].fd, in,-1 , response_path)) !=0 ) { +pseudo_debug (1 ,"failed to send response to client %d [%d]: %d (%s)\n" , + i, (int ) clients[i].pid, rc,strerror (errno)); + } + }else { + rc =1 ; + }free (response_path);return rc; }else {
@@ -423,10 +430,11 @@ pseudo_server_loop(void) { die_peacefully =1 ; }else {/* display this if not exiting */ -pseudo_debug (1 ,"%d messages handled in %.4f seconds\n" , +pseudo_debug (1 ,"%d messages handled in %.4f seconds, %d responses\n" , messages, (double ) message_time.tv_sec + - (double ) message_time.tv_usec /1000000.0 ); + (double ) message_time.tv_usec /1000000.0 , + responses); } } }else if (rc >0 ) {
@@ -439,11 +447,13 @@ pseudo_server_loop(void) {close_client (i); }else if (FD_ISSET (clients[i].fd, &reads)) {struct timeval tv1, tv2; -int op; +int rc;gettimeofday (&tv1,NULL ); - op =serve_client (i); + rc =serve_client (i);gettimeofday (&tv2,NULL ); ++messages; +if (rc ==0 ) + ++responses; message_time.tv_sec += (tv2.tv_sec - tv1.tv_sec); message_time.tv_usec += (tv2.tv_usec - tv1.tv_usec);if (message_time.tv_usec <0 ) {
Signature · 1 line
1.7.9.5
Quoted 33 lines · Peter Seebach
Unlike most of my submissions, this isn't patches against oe-core; rather, it's patches against pseudo, and if I can get some confirmation that they do what I think they do, and some review, I'm planning to make this into pseudo 1.5, and send a patch "soonish" to merge that into oe-core.
What this does: Fix a number of build performance issues. By far the largest change is actually not so much a problem with pseudo as a problem that pseudo can solve by brute force. Packaging systems (at least RPM and SMART) do a lot of
fsync()andfdatasync()calls. That usually implies flushing EVERYTHING that's been written, not just one specific file. And that, in turn, results in a severe performance hit.So, for instance, on one of my test workstations, this moves a
do_rootfswith about 1200 RPMs from about 22 minutes to about 4.5. Yeah.The other changes aren't as dramatic for that case, but have very significant performance impact for at least some workloads. The first is switching to using an in-memory database for the files database, dumping it to disk only when the pseudo daemon is idle or shutting down. This doesn't produce huge benefits in all cases, but for workloads with a lot of parallelism, it can produce a very noticeable reduction in how much pseudo slows things down.
The second is a fairly major protocol change. In short, with this patch, pseudo clients only wait for a server response when they need information from the server in order to continue. That's
OP_FSTAT,OP_STAT,OP_MAY_UNLINK, andOP_MKNOD. Everything else just silently assumes that it probably succeeded.How much does this matter? Between the protocol change and the memory DB, a trivial unpack of a tarball (lots of writes to the database, very few reads) can be about 4x faster. Removing stuff isn't much faster, but it might be a bit faster.
Quoted 6 lines
This is most noticeable, by far, when running more than one build, or when running builds while doing other things. It has a much smaller effect on builds with no shared state (compile time still dominates that), but even there I'm seeing decreases from ~83 minutes to ~64 from just the fsync and memory changes. Still waiting on my real test case (multiple simultaneous builds which need compiles) completing.
All sounds good to me. I've set the autobuilder away to test this set of changes.
Cheers,
Richard