Skip to contentSkip to conversation

Fix recursive mode -st on BUILDDIR setup

PATCH In review Alex Franco 4 participants 7 messages Sep 3, 2015 – Sep 24, 2015

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 Fix recursive mode -st on BUILDDIR setup

Removing recursive option from chmod -st on BUILDDIR as it would take very long on existing build directories

[YOCTO #7669]

---
 meta/classes/sanity.bbclass | 9 ++++++---
 scripts/oe-setup-builddir   | 5 ++++-
 2 files changed, 10 insertions(+), 4 deletions(-)
meta/classes/sanity.bbclass +6 -3
index 2864318..29bb619 100644
@@ -841,9 +841,12 @@ def check_sanity_everybuild(status, d):
     else:
         bb.utils.mkdirhier(tmpdir)
         # Remove setuid, setgid and sticky bits from TMPDIR
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISUID)
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISGID)
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISVTX)
+        try:
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISUID)
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISGID)
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISVTX)
+        except OSError:
+            bb.warn("Unable to chmod TMPDIR: %s" % tmpdir)
         with open(checkfile, "w") as f:
             f.write(tmpdir)
 
scripts/oe-setup-builddir +4 -1
index f5b7e4e..91bd86b 100755
@@ -24,7 +24,10 @@ if [ -z "$BUILDDIR" ]; then
 fi
 
 mkdir -p "$BUILDDIR/conf"
-chmod -R -st "$BUILDDIR" 
+
+# Attempting removal of sticky,setuid bits from BUILDDIR, BUILDDIR/conf
+chmod -st "$BUILDDIR" 2>/dev/null || echo "WARNING: unable to chmod $BUILDDIR"
+chmod -st "$BUILDDIR/conf" 2>/dev/null || echo "WARNING: unable to chmod $BUILDDIR/conf"
 
 if [ ! -d "$BUILDDIR" ]; then
     echo >&2 "Error: The builddir ($BUILDDIR) does not exist!"
Signature · 1 line
2.5.1
#1 lore raw
Quoted 2 lines · Alex Franco

Removing recursive option from chmod -st on BUILDDIR as it would take very long on existing build directories

Okay, so this *is* a problem others are also seeing ;-}

Quoted 6 lines
scripts/oe-setup-builddir +0 -0
index f5b7e4e..91bd86b 100755
@@ -24,7 +24,10 @@ if [ -z "$BUILDDIR" ]; then
 fi

scripts/oe-setup-builddir if [ -z "$BUILDDIR" ]; then · line 33

Quoted 10 lines
scripts/oe-setup-builddir +4 -1
 
 mkdir -p "$BUILDDIR/conf"
-chmod -R -st "$BUILDDIR" 
+
+# Attempting removal of sticky,setuid bits from BUILDDIR, BUILDDIR/conf
+chmod -st "$BUILDDIR" 2>/dev/null || echo "WARNING: unable to chmod $BUILDDIR"
+chmod -st "$BUILDDIR/conf" 2>/dev/null || echo "WARNING: unable to chmod $BUILDDIR/conf"
 
 if [ ! -d "$BUILDDIR" ]; then
     echo >&2 "Error: The builddir ($BUILDDIR) does not exist!"

What was the reasoning behind adding these operations on $BUILDDIR/conf before the check whether BUILDDIR exists and is a directory? Looks a bit fishy to me.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#2 lore raw
Quoted 20 lines · Patrick Ohly
Quoted 2 lines · Alex Franco

Removing recursive option from chmod -st on BUILDDIR as it would take very long on existing build directories

Okay, so this *is* a problem others are also seeing ;-}

Quoted 14 lines
scripts/oe-setup-builddir +4 -1
index f5b7e4e..91bd86b 100755
@@ -24,7 +24,10 @@ if [ -z "$BUILDDIR" ]; then
 fi
 
 mkdir -p "$BUILDDIR/conf"
-chmod -R -st "$BUILDDIR" 
+
+# Attempting removal of sticky,setuid bits from BUILDDIR, BUILDDIR/conf
+chmod -st "$BUILDDIR" 2>/dev/null || echo "WARNING: unable to chmod $BUILDDIR"
+chmod -st "$BUILDDIR/conf" 2>/dev/null || echo "WARNING: unable to chmod $BUILDDIR/conf"
 
Quoted 6 lines

scripts/oe-setup-builddir if [ -z "$BUILDDIR" ]; then · line 33

Quoted 2 lines
scripts/oe-setup-builddir +0 -0
 if [ ! -d "$BUILDDIR" ]; then
     echo >&2 "Error: The builddir ($BUILDDIR) does not exist!"

What was the reasoning behind adding these operations on $BUILDDIR/conf before the check whether BUILDDIR exists and is a directory? Looks a bit fishy to me.

Non-existent parent of $BUILDDIR is caught elsewhere, but pointing BUILDDIR to a file instead of a directory indeed leads to sub-optimal error reporting:

$ touch /tmp/foobar
$ . oe-init-build-env /tmp/foobar
mkdir: cannot create directory ‘/tmp/foobar’: Not a directory
chmod: cannot access ‘/tmp/foobar/conf’: Not a directory
Error: The builddir (/tmp/foobar) does not exist!

Not sure whether it's worth fixing, though. Better get this performance fix included quickly.

Signature · 6 lines
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
#3 lore raw
In reply to Patrick Ohly

I agree these operations should take place after those checks.

Alex

Quoted 25 lines · Patrick Ohly
Quoted 2 lines · Alex Franco

Removing recursive option from chmod -st on BUILDDIR as it would take very long on existing build directories

Okay, so this *is* a problem others are also seeing ;-}

Quoted 6 lines
scripts/oe-setup-builddir +0 -0
index f5b7e4e..91bd86b 100755
@@ -24,7 +24,10 @@ if [ -z "$BUILDDIR" ]; then
  fi

scripts/oe-setup-builddir if [ -z "$BUILDDIR" ]; then · line 33

Quoted 10 lines
scripts/oe-setup-builddir +4 -1
 
  mkdir -p "$BUILDDIR/conf"
-chmod -R -st "$BUILDDIR"
+
+# Attempting removal of sticky,setuid bits from BUILDDIR, BUILDDIR/conf
+chmod -st "$BUILDDIR" 2>/dev/null || echo "WARNING: unable to chmod $BUILDDIR"
+chmod -st "$BUILDDIR/conf" 2>/dev/null || echo "WARNING: unable to chmod $BUILDDIR/conf"
 
  if [ ! -d "$BUILDDIR" ]; then
      echo >&2 "Error: The builddir ($BUILDDIR) does not exist!"

What was the reasoning behind adding these operations on $BUILDDIR/conf before the check whether BUILDDIR exists and is a directory? Looks a bit fishy to me.

#4 lore raw

PATCH 1/1 sanity.bbclass: show warning when chmod fails

In reply to Alex Franco

From: Alex Franco <[email protected]>

* for some reason this part of:
http://patchwork.openembedded.org/patch/102561/
wasn't ever merged.

[YOCTO #7669]

---
 meta/classes/sanity.bbclass | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
meta/classes/sanity.bbclass +6 -3
index 2eb744f..34f8618 100644
@@ -839,9 +839,12 @@ def check_sanity_everybuild(status, d):
     else:
         bb.utils.mkdirhier(tmpdir)
         # Remove setuid, setgid and sticky bits from TMPDIR
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISUID)
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISGID)
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISVTX)
+        try:
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISUID)
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISGID)
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISVTX)
+        except OSError:
+            bb.warn("Unable to chmod TMPDIR: %s" % tmpdir)
         with open(checkfile, "w") as f:
             f.write(tmpdir)
 
Signature · 1 line
2.5.3
#5 lore raw

Re: [PATCH] sanity.bbclass: show warning when chmod fails

Quoted 22 lines · Martin Jansa

From: Alex Franco <[email protected]>

* for some reason this part of:
http://patchwork.openembedded.org/patch/102561/
wasn't ever merged.

[YOCTO #7669]

---
 meta/classes/sanity.bbclass | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
meta/classes/sanity.bbclass +0 -0
index 2eb744f..34f8618 100644
@@ -839,9 +839,12 @@ def check_sanity_everybuild(status, d):
     else:
         bb.utils.mkdirhier(tmpdir)
         # Remove setuid, setgid and sticky bits from TMPDIR

meta/classes/sanity.bbclass def check_sanity_everybuild(status, d): · line 848

Quoted 10 lines
meta/classes/sanity.bbclass +6 -3
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISUID)
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISGID)
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISVTX)
+        try:
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISUID)
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISGID)
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISVTX)
+        except OSError:
+            bb.warn("Unable to chmod TMPDIR: %s" % tmpdir)
 

I'd suggest showing the actual error, here. E.g. "Unable to chmod TMPDIR:
%s" % exc. (The error message from the exception usually includes the path on disk already, so probably don't need both tmpdir and exc).

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#6 lore raw

Re: [PATCH] sanity.bbclass: show warning when chmod fails

In reply to Martin Jansa
Quoted 5 lines · Martin Jansa

From: Alex Franco <[email protected]>

* for some reason this part of:
http://patchwork.openembedded.org/patch/102561/
wasn't ever merged.

I've changed the configuration on my jenkins servers, so that now it doesn't fail.

In case someone needs that as well then you need to use mode, uid and git parameters for mounting tmpfs, e.g.

none /home/jenkins/oe/world/shr-core/tmp-glibc/ tmpfs noatime,nodev,nosuid,size=74G,mode=777,users,exec,uid=3004,gid=3004 0 0

where 3004 is uid and gid of jenkins user which is running the bitbake.

This change is still needed, because following exception shows so much about bbclass sanity, that average user will get confused and eventually insane.

ERROR: Execution of event handler 'check_sanity_eventhandler' failed
Traceback (most recent call last):
File "check_sanity_eventhandler(e)", line 6, in check_sanity_eventhandler(e=<bb.event.SanityCheck object at 0x57ec3d0>)
File "sanity.bbclass", line 34, in check_sanity(sanity_data=<bb.data_smart.DataSmart object at 0x57d5e10>)
File "sanity.bbclass", line 155, in check_sanity_everybuild(status=<SanityStatus object at 0x57d5490>, d=<bb.data_smart.DataSmart object at 0x57d5e10>)
OSError: [Errno 1] Operation not permitted: '/home/jenkins/workspace/luneos-unstable/webos-ports/tmp-glibc'

ERROR: Command execution failed: Traceback (most recent call last):
File "/home/jenkins/workspace/luneos-unstable/webos-ports/bitbake/lib/bb/command.py", line 101, in runAsyncCommand
self.cooker.updateCache()
File "/home/jenkins/workspace/luneos-unstable/webos-ports/bitbake/lib/bb/cooker.py", line 1503, in updateCache
bb.event.fire(bb.event.SanityCheck(False), self.data)
File "/home/jenkins/workspace/luneos-unstable/webos-ports/bitbake/lib/bb/event.py", line 170, in fire
fire_class_handlers(event, d)
File "/home/jenkins/workspace/luneos-unstable/webos-ports/bitbake/lib/bb/event.py", line 109, in fire_class_handlers
execute_handler(name, handler, event, d)
File "/home/jenkins/workspace/luneos-unstable/webos-ports/bitbake/lib/bb/event.py", line 81, in execute_handler
ret = handler(event)
File "check_sanity_eventhandler(e)", line 6, in check_sanity_eventhandler
File "sanity.bbclass", line 34, in check_sanity
File "sanity.bbclass", line 155, in check_sanity_everybuild
OSError: [Errno 1] Operation not permitted: '/home/jenkins/workspace/luneos-unstable/webos-ports/tmp-glibc'

Summary: There were 2 ERROR messages shown, returning a non-zero exit code. Command exited with non-zero status 1

Quoted 32 lines

[YOCTO #7669]

---
 meta/classes/sanity.bbclass | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
meta/classes/sanity.bbclass +6 -3
index 2eb744f..34f8618 100644
@@ -839,9 +839,12 @@ def check_sanity_everybuild(status, d):
     else:
         bb.utils.mkdirhier(tmpdir)
         # Remove setuid, setgid and sticky bits from TMPDIR
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISUID)
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISGID)
-        os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISVTX)
+        try:
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISUID)
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISGID)
+            os.chmod(tmpdir, os.stat(tmpdir).st_mode & ~ stat.S_ISVTX)
+        except OSError:
+            bb.warn("Unable to chmod TMPDIR: %s" % tmpdir)
         with open(checkfile, "w") as f:
             f.write(tmpdir)
 
Signature · 1 line
2.5.3
Signature · 1 line
Martin 'JaMa' Jansa     jabber: [email protected]