sstate.bbclass: update .siginfo atime

PATCH Superseded Ed Bartosh 5 participants 12 messages Mar 1, 2017 – Mar 7, 2017

Share & tools

Ed Bartosh

author

2017-03-01 14:35 UTC #0 lore raw

PATCH 1/1 sstate.bbclass: update .siginfo atime

.siginfo files are not being accessed from local or NFS-mounted sstate mirrors when sstate package is installed, so their atime is not updated. If sstate mirror is cleaned based on access time, they get deleted, even though they are still being used.

Updated atime of .siginfo symlinks with 'touch -a'. This command dereferences symlinks pointing to the local mirror and updates atime of the .siginfo file on the mirror.

[YOCTO #10857]

Signed-off-by Ed Bartosh <[email protected]>

---
 meta/classes/sstate.bbclass | 2 ++
 1 file changed, 2 insertions(+)

meta/classes/sstate.bbclass +2 -0

index 465fdcf..aa95964 100644
@@ -753,6 +753,8 @@ sstate_unpack_package () {
 	[ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
 	[ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] || touch --no-dereference ${SSTATE_PKG}.sig
 	[ ! -w ${SSTATE_PKG}.siginfo ] || [ ! -e ${SSTATE_PKG}.siginfo ] || touch --no-dereference ${SSTATE_PKG}.siginfo
+	# update .siginfo atime on local/NFS mirror
+	[ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
 }
 
 BB_HASHCHECK_FUNCTION = "sstate_checkhashes"

Signature · 1 line

2.1.4

Richard Purdie

2017-03-04 10:38 UTC #1 lore raw

Quoted 9 lines · Ed Bartosh

.siginfo files are not being accessed from local or NFS-mounted sstate mirrors when sstate package is installed, so their atime is not updated. If sstate mirror is cleaned based on access time, they get deleted, even though they are still being used.
Updated atime of .siginfo symlinks with 'touch -a'. This command dereferences symlinks pointing to the local mirror and updates atime of the .siginfo file on the mirror.

Quoted 6 lines

[YOCTO #10857]
Signed-off-by Ed Bartosh <[email protected]>
---
meta/classes/sstate.bbclass | 2 ++
 1 file changed, 2 insertions(+)

https://autobuilder.yoctoproject.org/main/builders/nightly-x86-64/builds/1138/steps/Running%20ESDK%20Sanity%20Tests/logs/stdio

rpurdie@opensuse132:~> less /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/testsdkext/tmp/work/core2-64-poky-linux/meta-extsdk-toolchain/1.0-r0/temp/log.do_populate_sysroot_setscene.31530
DEBUG: Executing python function do_populate_sysroot_setscene
DEBUG: Executing shell function sstate_unpack_package
sysroot-destdir/
sysroot-destdir/sysroot-providers/
sysroot-destdir/sysroot-providers/meta-extsdk-toolchain
WARNING: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/testsdkext/tmp/work/core2-64-poky-linux/meta-extsdk-toolchain/1.0-r0/temp/run.sstate_unpack_package.31530:1 exit 1 from '[ -h /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64/build/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/testsdkext/sstate-cache/56/sstate:meta-extsdk-toolchain:core2-64-poky-linux:1.0:r0:core2-64:3:563a7c8c6206b04c5b117d6df021982b_populate_sysroot.tgz.siginfo ]'
DEBUG: Python function do_populate_sysroot_setscene finished

So it looks like this failure is from this patch somehow...

Cheers,

Richard

Burton, Ross

2017-03-06 11:15 UTC #2 lore raw

In reply to Ed Bartosh

Quoted 2 lines · Ed Bartosh

+ [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo

I think you need to end this with || true to eat the failure if the test fails.

Ross

Ed Bartosh

author

2017-03-06 12:51 UTC #3 lore raw

Quoted 7 lines · Ross

Quoted 2 lines · Ed Bartosh
+ [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
I think you need to end this with || true to eat the failure if the test fails.

Thank you for pointing out to this.
I guess it should work also if I move this line up. I'll send v2 soon.

--
Regards,
Ed

Ed Bartosh

author

2017-03-06 15:46 UTC #4 lore raw

PATCH v2 1/1 sstate.bbclass: update .siginfo atime

In reply to Burton, Ross

Updated atime of .siginfo symlinks with 'touch -a'. This command dereferences symlinks pointing to the local mirror and updates atime of the .siginfo file on the mirror.

[YOCTO #10857]

Signed-off-by Ed Bartosh <[email protected]>

---
 meta/classes/sstate.bbclass | 2 ++
 1 file changed, 2 insertions(+)

meta/classes/sstate.bbclass +2 -0

index 23dda08..e415f68 100644
@@ -747,6 +747,8 @@ python sstate_sign_package () {
 #
 sstate_unpack_package () {
 	tar -xvzf ${SSTATE_PKG}
+	# update .siginfo atime on local/NFS mirror
+	[ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
 	# Use "! -w ||" to return true for read only files
 	[ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
 	[ ! -w ${SSTATE_PKG}.sig ] || [ ! -e ${SSTATE_PKG}.sig ] || touch --no-dereference ${SSTATE_PKG}.sig

Signature · 1 line

2.1.4

Burton, Ross

2017-03-06 16:49 UTC #5 lore raw

Quoted 2 lines · Ed Bartosh

+ [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo

$ ( [ -h / ] && echo something ); echo $?
1

If the -h fails then this is still going to cause sstate_unpack_package to trap.

Ross

Ed Bartosh

author

2017-03-06 18:04 UTC #6 lore raw

Quoted 10 lines · Ross

Quoted 2 lines · Ed Bartosh
+ [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
$ ( [ -h / ] && echo something ); echo $?
1
If the -h fails then this is still going to cause sstate_unpack_package to trap.

I don't think it will as '&&' and '||' lists are among other special cases where shell doesn't exits on error.

Here is a proof:

$ cat test.sh

set -e

trap 'ouch!' ERR

foo() {
[ -h /etc/fstab ] && echo 'fstab is a symlink' echo 'end of foo'
}

foo
echo $?

$ $ sh -e test.sh
end of foo
0

--
Regards,
Ed

Patrick Ohly

2017-03-06 19:31 UTC #7 lore raw

Quoted 6 lines · Ed Bartosh

Quoted 5 lines · Ross
Quoted 2 lines · Ed Bartosh
+ [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo

Quoted 8 lines

Quoted 5 lines
$ ( [ -h / ] && echo something ); echo $?
1
If the -h fails then this is still going to cause sstate_unpack_package to trap.
I don't think it will as '&&' and '||' lists are among other special cases where shell doesn't exits on error.

True, but if the clause happens to be the last one in a function or script (for example, after restructuring code), then the return code of that becomes non-zero, which isn't intended.

$ cat test.sh 
foo() {
    [ -h /foobar ] && echo 'fstab is a symlink'
}

foo
echo $?

$ bash -xe test.sh 
+ set -e
+ foo
+ '[' -h /foobar ']'
$ echo $?
1

Making the intend explicit with || true avoids that.

Signature · 6 lines

Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Ed Bartosh

author

2017-03-06 20:11 UTC #8 lore raw

Quoted 13 lines · Patrick Ohly

Quoted 12 lines · Ed Bartosh
Quoted 10 lines · Ross
Quoted 2 lines · Ed Bartosh
+ [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
$ ( [ -h / ] && echo something ); echo $?
1
If the -h fails then this is still going to cause sstate_unpack_package to trap.

Quoted 6 lines

Quoted 2 lines
I don't think it will as '&&' and '||' lists are among other special cases where shell doesn't exits on error.
True, but if the clause happens to be the last one in a function or script (for example, after restructuring code), then the return code of that becomes non-zero, which isn't intended.

Exactly because of this I moved my changes up in the function in v2.

Quoted 27 lines

$ cat test.sh 
foo() {
    [ -h /foobar ] && echo 'fstab is a symlink'
}

foo
echo $?

$ bash -xe test.sh 
+ set -e
+ foo
+ '[' -h /foobar ']'
$ echo $?
1

Making the intend explicit with || true avoids that.

Signature · 6 lines

Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Signature · 3 lines

--
Regards,
Ed

Patrick Ohly

2017-03-06 21:46 UTC #9 lore raw

Quoted 14 lines · Ed Bartosh

Quoted 13 lines · Patrick Ohly
Quoted 12 lines · Ed Bartosh
Quoted 10 lines · Ross
Quoted 2 lines · Ed Bartosh
+ [ -h ${SSTATE_PKG}.siginfo ] && touch -a ${SSTATE_PKG}.siginfo
$ ( [ -h / ] && echo something ); echo $?
1
If the -h fails then this is still going to cause sstate_unpack_package to trap.

Quoted 8 lines

Quoted 6 lines
Quoted 2 lines
I don't think it will as '&&' and '||' lists are among other special cases where shell doesn't exits on error.
True, but if the clause happens to be the last one in a function or script (for example, after restructuring code), then the return code of that becomes non-zero, which isn't intended.
Exactly because of this I moved my changes up in the function in v2.

And someone else might accidentally move it down again, without understanding the consequences. It's simply safer to be explicit, IMHO. Not worth a v3, but perhaps in the future.

Signature · 6 lines

Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.

Peter Kjellerstedt

2017-03-07 11:29 UTC #10 lore raw

Quoted 33 lines

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Patrick Ohly
Sent: den 6 mars 2017 22:46
To: [email protected]
Cc: OE-core
Subject: Re: [OE-core] [PATCH v2] sstate.bbclass: update .siginfo atime
Quoted 4 lines · Ed Bartosh
Quoted 3 lines · Patrick Ohly
Quoted 2 lines · Ed Bartosh
Quoted 1 line · Ross
On 6 March 2017 at 15:31, Ed Bartosh
Quoted 2 lines · <[email protected]> wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
+ [ -h ${SSTATE_PKG}.siginfo ] && touch -a
${SSTATE_PKG}.siginfo
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line
$ ( [ -h / ] && echo something ); echo $?
1
If the -h fails then this is still going to cause
sstate_unpack_package to
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
trap.
I don't think it will as '&&' and '||' lists are among other
special cases where
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
shell doesn't exits on error.
True, but if the clause happens to be the last one in a function or script (for example, after restructuring code), then the return

Quoted 8 lines

code of
Quoted 3 lines
Quoted 1 line
that becomes non-zero, which isn't intended.
Exactly because of this I moved my changes up in the function in v2.
And someone else might accidentally move it down again, without understanding the consequences. It's simply safer to be explicit, IMHO. Not worth a v3, but perhaps in the future.

This is why our shell expert always recommends that tests like this should be written this way instead:

     [ ! -h /foobar ] || echo 'fstab is a symlink'

I.e., reverse the first test and always use || rather than &&.

//Peter

Burton, Ross

2017-03-06 20:49 UTC #11 lore raw

In reply to Ed Bartosh

Quoted 4 lines · Ed Bartosh

I don't think it will as '&&' and '||' lists are among other special cases where
shell doesn't exits on error.

Ah, fair enough. Another quirk of sh I wasn't aware of :)

Ross