Skip to contentSkip to conversation

V3: Replace retry logic in useradd with flock

PATCH In review [email protected] 4 participants 10 messages Feb 23, 2016 – Feb 24, 2016

lore
Share & tools
lore

[email protected]

author
#0 lore raw

cover V3: Replace retry logic in useradd with flock

From: Kai Kang <[email protected]>

V3:

* remove checks before operations such as useradd/userdel etc.

The following changes since commit ea8c34e976c11757ed9869c51b48f39050e25708:

libnewt: Fix build with PIE flags (2016-02-21 09:32:43 +0000)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib kangkai/flockv3
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kangkai/flockv3

Kai Kang (1):
useradd_base.bbclass: fix simultaneous with flock

 meta/classes/useradd.bbclass      |   6 +-
 meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
 2 files changed, 40 insertions(+), 152 deletions(-)
Signature · 1 line
2.6.1
author
#1 lore raw

PATCH 1/1 useradd_base.bbclass: fix simultaneous with flock

From: Kai Kang <[email protected]>

When perform useradd during populate sysroot, it locks files passwd.lock and group.lock at same time. And then it meets a dead lock issue randomly.

Use flock to reslove it by using an universal lock file for all the user and group related operations.

[YOCTO #9022]

Signed-off-by Kai Kang <[email protected]>
---
 meta/classes/useradd.bbclass      |   6 +-
 meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
 2 files changed, 40 insertions(+), 152 deletions(-)
meta/classes/useradd.bbclass +3 -3
index c960656..0a6f2be 100644
@@ -57,7 +57,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
 	opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
 	remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
 	while test "x$opts" != "x"; do
-		perform_groupadd "$SYSROOT" "$OPT $opts" 10
+		perform_groupadd "$SYSROOT" "$OPT $opts"
 		if test "x$opts" = "x$remaining"; then
 			break
 		fi

@@ -73,7 +73,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
 	opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
 	remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
 	while test "x$opts" != "x"; do
-		perform_useradd "$SYSROOT" "$OPT $opts" 10
+		perform_useradd "$SYSROOT" "$OPT $opts"
 		if test "x$opts" = "x$remaining"; then
 			break
 		fi

@@ -89,7 +89,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
 	opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
 	remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
 	while test "x$opts" != "x"; do
-		perform_groupmems "$SYSROOT" "$OPT $opts" 10
+		perform_groupmems "$SYSROOT" "$OPT $opts"
 		if test "x$opts" = "x$remaining"; then
 			break
 		fi
meta/classes/useradd_base.bbclass +37 -149

254 lines not on this page · raw message

Signature · 1 line
2.6.1
#2 lore raw

Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock

Patch subject is unclear. Patch itself looks ok

Quoted 319 lines · On Mon, Feb 22, 2016 at 9:45 PM, <[email protected]> wrote:

From: Kai Kang <[email protected]>

When perform useradd during populate sysroot, it locks files passwd.lock and group.lock at same time. And then it meets a dead lock issue randomly.

Use flock to reslove it by using an universal lock file for all the user and group related operations.

[YOCTO #9022]

Signed-off-by Kai Kang <[email protected]>
---
 meta/classes/useradd.bbclass      |   6 +-
 meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
 2 files changed, 40 insertions(+), 152 deletions(-)
meta/classes/useradd.bbclass +3 -3
index c960656..0a6f2be 100644
@@ -57,7 +57,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
        opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
        remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
        while test "x$opts" != "x"; do
-               perform_groupadd "$SYSROOT" "$OPT $opts" 10
+               perform_groupadd "$SYSROOT" "$OPT $opts"
                if test "x$opts" = "x$remaining"; then
                        break
                fi

@@ -73,7 +73,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
        opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
        remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
        while test "x$opts" != "x"; do
-               perform_useradd "$SYSROOT" "$OPT $opts" 10
+               perform_useradd "$SYSROOT" "$OPT $opts"
                if test "x$opts" = "x$remaining"; then
                        break
                fi

@@ -89,7 +89,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
        opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
        remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
        while test "x$opts" != "x"; do
-               perform_groupmems "$SYSROOT" "$OPT $opts" 10
+               perform_groupmems "$SYSROOT" "$OPT $opts"
                if test "x$opts" = "x$remaining"; then
                        break
                fi
meta/classes/useradd_base.bbclass +37 -150
index ab3cd35..7c1cc13 100644
@@ -4,7 +4,7 @@
 
 # The following functions basically have similar logic.
 # *) Perform necessary checks before invoking the actual command
-# *) Invoke the actual command, make retries if necessary
+# *) Invoke the actual command with flock
 # *) Error out if an error occurs.
 
 # Note that before invoking these functions, make sure the global variable

@@ -12,65 +12,32 @@
 
 perform_groupadd () {
        local rootdir="$1"
-       local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
+       local opts=`echo $2 | sed s/\'/\"/g`
+       bbnote "${PN}: Performing groupadd with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupadd $opts\' || true
        local groupname=`echo "$opts" | awk '{ print $NF }'`
-       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+       local group_exists=`grep "^$groupname:" $rootdir/etc/group`
        if test "x$group_exists" = "x"; then
-               local count=0
-               while true; do
-                       eval $PSEUDO groupadd $opts || true
-                       group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
-                       if test "x$group_exists" = "x"; then
-                               bbwarn "${PN}: groupadd command did not succeed. Retrying..."
-                       else
-                               break
-                       fi
-                       count=`expr $count + 1`
-                       if test $count = $retries; then
-                               bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
-                       fi
-                        sleep $count
-               done
-       else
-               bbnote "${PN}: group $groupname already exists, not re-creating it"
+               bbfatal "${PN}: groupadd command did not succeed."
        fi
 }
 
 perform_useradd () {
        local rootdir="$1"
-       local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
+       local opts=`echo $2 | sed s/\'/\"/g`
+       bbnote "${PN}: Performing useradd with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c  \'$PSEUDO useradd $opts\' || true
        local username=`echo "$opts" | awk '{ print $NF }'`
-       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+       local user_exists=`grep "^$username:" $rootdir/etc/passwd`
        if test "x$user_exists" = "x"; then
-              local count=0
-              while true; do
-                      eval $PSEUDO useradd $opts || true
-                      user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
-                      if test "x$user_exists" = "x"; then
-                              bbwarn "${PN}: useradd command did not succeed. Retrying..."
-                      else
-                              break
-                      fi
-                      count=`expr $count + 1`
-                      if test $count = $retries; then
-                               bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
-                      fi
-                      sleep $count
-              done
-       else
-               bbnote "${PN}: user $username already exists, not re-creating it"
+               bbfatal "${PN}: useradd command did not succeed."
        fi
 }
 
 perform_groupmems () {
        local rootdir="$1"
        local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
+       bbnote "${PN}: Performing groupmems with [$opts]"
        local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
        local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
        bbnote "${PN}: Running groupmems command with group $groupname and user $username"

@@ -82,30 +49,13 @@ perform_groupmems () {
                gshadow="no"
                touch $rootdir${sysconfdir}/gshadow
        fi
-       local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
+
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmems $opts\' || true
+       local mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group`
        if test "x$mem_exists" = "x"; then
-               local count=0
-               while true; do
-                       eval $PSEUDO groupmems $opts || true
-                       mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
-                       if test "x$mem_exists" = "x"; then
-                               bbwarn "${PN}: groupmems command did not succeed. Retrying..."
-                       else
-                               break
-                       fi
-                       count=`expr $count + 1`
-                       if test $count = $retries; then
-                               if test "x$gshadow" = "xno"; then
-                                       rm -f $rootdir${sysconfdir}/gshadow
-                                       rm -f $rootdir${sysconfdir}/gshadow-
-                               fi
-                               bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
-                       fi
-                       sleep $count
-               done
-       else
-               bbnote "${PN}: group $groupname already contains $username, not re-adding it"
+               bbfatal "${PN}: groupmems command did not succeed."
        fi
+
        if test "x$gshadow" = "xno"; then
                rm -f $rootdir${sysconfdir}/gshadow
                rm -f $rootdir${sysconfdir}/gshadow-

@@ -115,56 +65,24 @@ perform_groupmems () {
 perform_groupdel () {
        local rootdir="$1"
        local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
+       bbnote "${PN}: Performing groupdel with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupdel $opts\' || true
        local groupname=`echo "$opts" | awk '{ print $NF }'`
-       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+       local group_exists=`grep "^$groupname:" $rootdir/etc/group`
        if test "x$group_exists" != "x"; then
-               local count=0
-               while true; do
-                       eval $PSEUDO groupdel $opts || true
-                       group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
-                       if test "x$group_exists" != "x"; then
-                               bbwarn "${PN}: groupdel command did not succeed. Retrying..."
-                       else
-                               break
-                       fi
-                       count=`expr $count + 1`
-                       if test $count = $retries; then
-                               bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
-                       fi
-                       sleep $count
-               done
-       else
-               bbnote "${PN}: group $groupname doesn't exist, not removing it"
+               bbfatal "${PN}: groupdel command did not succeed."
        fi
 }
 
 perform_userdel () {
        local rootdir="$1"
        local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
+       bbnote "${PN}: Performing userdel with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO userdel $opts\' || true
        local username=`echo "$opts" | awk '{ print $NF }'`
-       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+       local user_exists=`grep "^$username:" $rootdir/etc/passwd`
        if test "x$user_exists" != "x"; then
-              local count=0
-              while true; do
-                      eval $PSEUDO userdel $opts || true
-                      user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
-                      if test "x$user_exists" != "x"; then
-                              bbwarn "${PN}: userdel command did not succeed. Retrying..."
-                      else
-                              break
-                      fi
-                      count=`expr $count + 1`
-                      if test $count = $retries; then
-                               bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
-                      fi
-                      sleep $count
-              done
-       else
-               bbnote "${PN}: user $username doesn't exist, not removing it"
+               bbfatal "${PN}: userdel command did not succeed."
        fi
 }
 

@@ -172,59 +90,29 @@ perform_groupmod () {
        # Other than the return value of groupmod, there's no simple way to judge whether the command
        # succeeds, so we disable -e option temporarily
        set +e
+
        local rootdir="$1"
        local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
-       local groupname=`echo "$opts" | awk '{ print $NF }'`
-       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
-       if test "x$group_exists" != "x"; then
-               local count=0
-               while true; do
-                       eval $PSEUDO groupmod $opts
-                       if test $? != 0; then
-                               bbwarn "${PN}: groupmod command did not succeed. Retrying..."
-                       else
-                               break
-                       fi
-                       count=`expr $count + 1`
-                       if test $count = $retries; then
-                               bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
-                       fi
-                       sleep $count
-               done
-       else
-               bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
+       bbnote "${PN}: Performing groupmod with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmod $opts\'
+       if test $? != 0; then
+               bbwarn "${PN}: groupmod command did not succeed."
        fi
+
        set -e
 }
 
 perform_usermod () {
        # Same reason with groupmod, temporarily disable -e option
        set +e
+
        local rootdir="$1"
        local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
-       local username=`echo "$opts" | awk '{ print $NF }'`
-       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
-       if test "x$user_exists" != "x"; then
-              local count=0
-              while true; do
-                      eval $PSEUDO usermod $opts
-                      if test $? != 0; then
-                              bbwarn "${PN}: usermod command did not succeed. Retrying..."
-                      else
-                              break
-                      fi
-                      count=`expr $count + 1`
-                      if test $count = $retries; then
-                               bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
-                      fi
-                      sleep $count
-              done
-       else
-               bbwarn "${PN}: user $username doesn't exist, unable to modify it"
+       bbnote "${PN}: Performing usermod with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO usermod $opts\'
+       if test $? != 0; then
+               bbfatal "${PN}: usermod command did not succeed."
        fi
+
        set -e
 }
--

2.6.1

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

author
#3 lore raw

Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock

Quoted 1 line · Khem Raj

Patch subject is unclear. Patch itself looks ok

How about use the title of Yocto #9022:

useradd_base.bbclass: replace retry logic with flock

Thanks,
Kai

Quoted 321 lines · On Mon, Feb 22, 2016 at 9:45 PM, <[email protected]> wrote:
Quoted 319 lines

From: Kai Kang <[email protected]>

When perform useradd during populate sysroot, it locks files passwd.lock and group.lock at same time. And then it meets a dead lock issue randomly.

Use flock to reslove it by using an universal lock file for all the user and group related operations.

[YOCTO #9022]

Signed-off-by Kai Kang <[email protected]>
---
  meta/classes/useradd.bbclass      |   6 +-
  meta/classes/useradd_base.bbclass | 186 ++++++++------------------------------
  2 files changed, 40 insertions(+), 152 deletions(-)
meta/classes/useradd.bbclass +3 -3
index c960656..0a6f2be 100644
@@ -57,7 +57,7 @@ if test "x`echo $GROUPADD_PARAM | tr -d '[:space:]'`" != "x"; then
         opts=`echo "$GROUPADD_PARAM" | cut -d ';' -f 1`
         remaining=`echo "$GROUPADD_PARAM" | cut -d ';' -f 2-`
         while test "x$opts" != "x"; do
-               perform_groupadd "$SYSROOT" "$OPT $opts" 10
+               perform_groupadd "$SYSROOT" "$OPT $opts"
                 if test "x$opts" = "x$remaining"; then
                         break
                 fi

@@ -73,7 +73,7 @@ if test "x`echo $USERADD_PARAM | tr -d '[:space:]'`" != "x"; then
         opts=`echo "$USERADD_PARAM" | cut -d ';' -f 1`
         remaining=`echo "$USERADD_PARAM" | cut -d ';' -f 2-`
         while test "x$opts" != "x"; do
-               perform_useradd "$SYSROOT" "$OPT $opts" 10
+               perform_useradd "$SYSROOT" "$OPT $opts"
                 if test "x$opts" = "x$remaining"; then
                         break
                 fi

@@ -89,7 +89,7 @@ if test "x`echo $GROUPMEMS_PARAM | tr -d '[:space:]'`" != "x"; then
         opts=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 1`
         remaining=`echo "$GROUPMEMS_PARAM" | cut -d ';' -f 2-`
         while test "x$opts" != "x"; do
-               perform_groupmems "$SYSROOT" "$OPT $opts" 10
+               perform_groupmems "$SYSROOT" "$OPT $opts"
                 if test "x$opts" = "x$remaining"; then
                         break
                 fi
meta/classes/useradd_base.bbclass +37 -150
index ab3cd35..7c1cc13 100644
@@ -4,7 +4,7 @@
 
  # The following functions basically have similar logic.
  # *) Perform necessary checks before invoking the actual command
-# *) Invoke the actual command, make retries if necessary
+# *) Invoke the actual command with flock
  # *) Error out if an error occurs.
 
  # Note that before invoking these functions, make sure the global variable

@@ -12,65 +12,32 @@
 
  perform_groupadd () {
         local rootdir="$1"
-       local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing groupadd with [$opts] and $retries times of retry"
+       local opts=`echo $2 | sed s/\'/\"/g`
+       bbnote "${PN}: Performing groupadd with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupadd $opts\' || true
         local groupname=`echo "$opts" | awk '{ print $NF }'`
-       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+       local group_exists=`grep "^$groupname:" $rootdir/etc/group`
         if test "x$group_exists" = "x"; then
-               local count=0
-               while true; do
-                       eval $PSEUDO groupadd $opts || true
-                       group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
-                       if test "x$group_exists" = "x"; then
-                               bbwarn "${PN}: groupadd command did not succeed. Retrying..."
-                       else
-                               break
-                       fi
-                       count=`expr $count + 1`
-                       if test $count = $retries; then
-                               bbfatal "${PN}: Tried running groupadd command $retries times without success, giving up"
-                       fi
-                        sleep $count
-               done
-       else
-               bbnote "${PN}: group $groupname already exists, not re-creating it"
+               bbfatal "${PN}: groupadd command did not succeed."
         fi
  }
 
  perform_useradd () {
         local rootdir="$1"
-       local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing useradd with [$opts] and $retries times of retry"
+       local opts=`echo $2 | sed s/\'/\"/g`
+       bbnote "${PN}: Performing useradd with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c  \'$PSEUDO useradd $opts\' || true
         local username=`echo "$opts" | awk '{ print $NF }'`
-       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+       local user_exists=`grep "^$username:" $rootdir/etc/passwd`
         if test "x$user_exists" = "x"; then
-              local count=0
-              while true; do
-                      eval $PSEUDO useradd $opts || true
-                      user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
-                      if test "x$user_exists" = "x"; then
-                              bbwarn "${PN}: useradd command did not succeed. Retrying..."
-                      else
-                              break
-                      fi
-                      count=`expr $count + 1`
-                      if test $count = $retries; then
-                               bbfatal "${PN}: Tried running useradd command $retries times without success, giving up"
-                      fi
-                      sleep $count
-              done
-       else
-               bbnote "${PN}: user $username already exists, not re-creating it"
+               bbfatal "${PN}: useradd command did not succeed."
         fi
  }
 
  perform_groupmems () {
         local rootdir="$1"
         local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing groupmems with [$opts] and $retries times of retry"
+       bbnote "${PN}: Performing groupmems with [$opts]"
         local groupname=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-g" || $i == "--group") print $(i+1) }'`
         local username=`echo "$opts" | awk '{ for (i = 1; i < NF; i++) if ($i == "-a" || $i == "--add") print $(i+1) }'`
         bbnote "${PN}: Running groupmems command with group $groupname and user $username"

@@ -82,30 +49,13 @@ perform_groupmems () {
                 gshadow="no"
                 touch $rootdir${sysconfdir}/gshadow
         fi
-       local mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
+
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmems $opts\' || true
+       local mem_exists=`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group`
         if test "x$mem_exists" = "x"; then
-               local count=0
-               while true; do
-                       eval $PSEUDO groupmems $opts || true
-                       mem_exists="`grep "^$groupname:[^:]*:[^:]*:\([^,]*,\)*$username\(,[^,]*\)*" $rootdir/etc/group || true`"
-                       if test "x$mem_exists" = "x"; then
-                               bbwarn "${PN}: groupmems command did not succeed. Retrying..."
-                       else
-                               break
-                       fi
-                       count=`expr $count + 1`
-                       if test $count = $retries; then
-                               if test "x$gshadow" = "xno"; then
-                                       rm -f $rootdir${sysconfdir}/gshadow
-                                       rm -f $rootdir${sysconfdir}/gshadow-
-                               fi
-                               bbfatal "${PN}: Tried running groupmems command $retries times without success, giving up"
-                       fi
-                       sleep $count
-               done
-       else
-               bbnote "${PN}: group $groupname already contains $username, not re-adding it"
+               bbfatal "${PN}: groupmems command did not succeed."
         fi
+
         if test "x$gshadow" = "xno"; then
                 rm -f $rootdir${sysconfdir}/gshadow
                 rm -f $rootdir${sysconfdir}/gshadow-

@@ -115,56 +65,24 @@ perform_groupmems () {
  perform_groupdel () {
         local rootdir="$1"
         local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing groupdel with [$opts] and $retries times of retry"
+       bbnote "${PN}: Performing groupdel with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupdel $opts\' || true
         local groupname=`echo "$opts" | awk '{ print $NF }'`
-       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
+       local group_exists=`grep "^$groupname:" $rootdir/etc/group`
         if test "x$group_exists" != "x"; then
-               local count=0
-               while true; do
-                       eval $PSEUDO groupdel $opts || true
-                       group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
-                       if test "x$group_exists" != "x"; then
-                               bbwarn "${PN}: groupdel command did not succeed. Retrying..."
-                       else
-                               break
-                       fi
-                       count=`expr $count + 1`
-                       if test $count = $retries; then
-                               bbfatal "${PN}: Tried running groupdel command $retries times without success, giving up"
-                       fi
-                       sleep $count
-               done
-       else
-               bbnote "${PN}: group $groupname doesn't exist, not removing it"
+               bbfatal "${PN}: groupdel command did not succeed."
         fi
  }
 
  perform_userdel () {
         local rootdir="$1"
         local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing userdel with [$opts] and $retries times of retry"
+       bbnote "${PN}: Performing userdel with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO userdel $opts\' || true
         local username=`echo "$opts" | awk '{ print $NF }'`
-       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
+       local user_exists=`grep "^$username:" $rootdir/etc/passwd`
         if test "x$user_exists" != "x"; then
-              local count=0
-              while true; do
-                      eval $PSEUDO userdel $opts || true
-                      user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
-                      if test "x$user_exists" != "x"; then
-                              bbwarn "${PN}: userdel command did not succeed. Retrying..."
-                      else
-                              break
-                      fi
-                      count=`expr $count + 1`
-                      if test $count = $retries; then
-                               bbfatal "${PN}: Tried running userdel command $retries times without success, giving up"
-                      fi
-                      sleep $count
-              done
-       else
-               bbnote "${PN}: user $username doesn't exist, not removing it"
+               bbfatal "${PN}: userdel command did not succeed."
         fi
  }
 

@@ -172,59 +90,29 @@ perform_groupmod () {
         # Other than the return value of groupmod, there's no simple way to judge whether the command
         # succeeds, so we disable -e option temporarily
         set +e
+
         local rootdir="$1"
         local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing groupmod with [$opts] and $retries times of retry"
-       local groupname=`echo "$opts" | awk '{ print $NF }'`
-       local group_exists="`grep "^$groupname:" $rootdir/etc/group || true`"
-       if test "x$group_exists" != "x"; then
-               local count=0
-               while true; do
-                       eval $PSEUDO groupmod $opts
-                       if test $? != 0; then
-                               bbwarn "${PN}: groupmod command did not succeed. Retrying..."
-                       else
-                               break
-                       fi
-                       count=`expr $count + 1`
-                       if test $count = $retries; then
-                               bbfatal "${PN}: Tried running groupmod command $retries times without success, giving up"
-                       fi
-                       sleep $count
-               done
-       else
-               bbwarn "${PN}: group $groupname doesn't exist, unable to modify it"
+       bbnote "${PN}: Performing groupmod with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO groupmod $opts\'
+       if test $? != 0; then
+               bbwarn "${PN}: groupmod command did not succeed."
         fi
+
         set -e
  }
 
  perform_usermod () {
         # Same reason with groupmod, temporarily disable -e option
         set +e
+
         local rootdir="$1"
         local opts="$2"
-       local retries="$3"
-       bbnote "${PN}: Performing usermod with [$opts] and $retries times of retry"
-       local username=`echo "$opts" | awk '{ print $NF }'`
-       local user_exists="`grep "^$username:" $rootdir/etc/passwd || true`"
-       if test "x$user_exists" != "x"; then
-              local count=0
-              while true; do
-                      eval $PSEUDO usermod $opts
-                      if test $? != 0; then
-                              bbwarn "${PN}: usermod command did not succeed. Retrying..."
-                      else
-                              break
-                      fi
-                      count=`expr $count + 1`
-                      if test $count = $retries; then
-                               bbfatal "${PN}: Tried running usermod command $retries times without success, giving up"
-                      fi
-                      sleep $count
-              done
-       else
-               bbwarn "${PN}: user $username doesn't exist, unable to modify it"
+       bbnote "${PN}: Performing usermod with [$opts]"
+       eval flock -x -w 100 $rootdir${sysconfdir} -c \'$PSEUDO usermod $opts\'
+       if test $? != 0; then
+               bbfatal "${PN}: usermod command did not succeed."
         fi
+
         set -e
  }
--

2.6.1

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Signature · 2 lines
Regards,
Neil | Kai Kang
#4 lore raw

Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock

Quoted 6 lines · Kang Kai
Quoted 1 line · Khem Raj

Patch subject is unclear. Patch itself looks ok

How about use the title of Yocto #9022:

useradd_base.bbclass: replace retry logic with flock

Sounds good to me, thanks!

Richard

author
#5 lore raw

Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock

Quoted 7 lines · Richard Purdie
Quoted 5 lines · Kang Kai
Quoted 1 line · Khem Raj

Patch subject is unclear. Patch itself looks ok

How about use the title of Yocto #9022:

useradd_base.bbclass: replace retry logic with flock

Sounds good to me, thanks!

Updated the git repo:

git://git.pokylinux.org/poky-contrib kangkai/flockv3

Thanks.

--Kai

Quoted 3 lines

Richard

Signature · 2 lines
Regards,
Neil | Kai Kang
#6 lore raw

Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock

In reply to [email protected]
Quoted 7 lines · On 23 February 2016 at 02:45, <[email protected]> wrote:

When perform useradd during populate sysroot, it locks files passwd.lock and group.lock at same time. And then it meets a dead lock issue randomly.

Use flock to reslove it by using an universal lock file for all the user and group related operations.

I got an error when openssh's do_install was running useradds to the sysroot:

| DEBUG: Executing shell function useradd_sysroot
| Running useradd commands...
|

/data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/run.useradd_sysroot.13825:
264: local: /data/poky-master/tmp-glibc/sysroots/intel-core2-32: bad
variable name
| WARNING: exit code 2 from a shell command.
| ERROR: Function failed: useradd_sysroot (log file is located at /data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/log.do_install.13825)
ERROR: Task 2939 (/home/ross/Yocto/poky/meta/recipes-connectivity/openssh/ openssh_7.1p2.bb, do_install) failed with exit code '1'

    262 perform_useradd() {
    263         local rootdir="$1"
    264         local opts=`echo $2 | sed s/\'/\"/g`
    265         bbnote "openssh: Performing useradd with [$opts]"

Ross

#7 lore raw

Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock

Quoted 15 lines · Ross

I got an error when openssh's do_install was running useradds to the sysroot:

| DEBUG: Executing shell function useradd_sysroot
| Running useradd commands...
|

/data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/run.useradd_sysroot.13825:
264: local: /data/poky-master/tmp-glibc/sysroots/intel-core2-32: bad
variable name
| WARNING: exit code 2 from a shell command.
| ERROR: Function failed: useradd_sysroot (log file is located at /data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/log.do_install.13825)
ERROR: Task 2939 (/home/ross/Yocto/poky/meta/recipes-connectivity/openssh/ openssh_7.1p2.bb, do_install) failed with exit code '1'

Quoted 5 lines
    262 perform_useradd() {
    263         local rootdir="$1"
    264         local opts=`echo $2 | sed s/\'/\"/g`
    265         bbnote "openssh: Performing useradd with [$opts]"

The *amazing* BB_VERBOSE_LOGS option (set to 1 in local.conf to get all shell scripts to do set -x) tells me this is due to missing quotes around the opts assignment:

+ local opts=--root /data/poky-master/tmp-glibc/sysroots/intel-core2-32
--system --no-create-home --home-dir /var/run/sshd --shell /bin/false
--user-group sshd /data/poky-master/tmp-glibc/work/core2-32-poky-linux/openssh/7.1p2-r0/temp/run.useradd_sysroot.16570:
264: local: /data/poky-master/tmp-glibc/sysroots/intel-core2-32: bad
variable name

Ross

#8 lore raw

Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock

Quoted 4 lines · Ross

The *amazing* BB_VERBOSE_LOGS option (set to 1 in local.conf to get all shell scripts to do set -x) tells me this is due to missing quotes around the opts assignment:

Why do some of the opts assignments just do opts=$2, and others do a sed?

Ross

author
#9 lore raw

Re: [PATCH 1/1] useradd_base.bbclass: fix simultaneous with flock

Quoted 9 lines · Ross

On 23 February 2016 at 17:01, Burton, Ross <[email protected] <mailto:[email protected]>> wrote:

The *amazing* BB_VERBOSE_LOGS option (set to 1 in local.conf to get all shell scripts to do set -x) tells me this is due to missing quotes around the opts assignment:

Why do some of the opts assignments just do opts=$2, and others do a sed?

In the previous version, there are:

local opts="$2"
...

opts=echo $opts | sed s/\'/\"/g

And I want to refactor them but lost the double quotes.

Only do sed operations for useradd and groupadd that they may be passed in "opts" with space but others didn't.

I'll restore to previous version.

Sorry for inconvenience that I built pass on Ubuntu 14.04 so didn't find this error.

--Kai

Quoted 2 lines

Ross

Signature · 2 lines
Regards,
Neil | Kai Kang