image.bbclass: Do not expand variables in IMAGE_CMD vardepsexclude

PATCH In review Jonathan Liu 2 participants 5 messages Sep 11, 2016 – Sep 13, 2016

Share & tools

Jonathan Liu

author

2016-09-11 12:14 UTC #0 lore raw

PATCH 1/1 image.bbclass: Do not expand variables in IMAGE_CMD vardepsexclude

This fixes tashhash mismatch errors creating images when IMAGE_CMD references a variable whose value is always changing even though the variable is specified in IMAGE_CMD vardepsexclude.

Signed-off-by Jonathan Liu <[email protected]>

---
 meta/classes/image.bbclass | 4 ++++
 1 file changed, 4 insertions(+)

meta/classes/image.bbclass +4 -0

index 8c8b8a2..f85ea86 100644
@@ -441,6 +441,10 @@ python () {
         localdata.delVar('DATETIME')
         localdata.delVar('TMPDIR')
 
+        vardepsexclude = d.getVarFlag('IMAGE_CMD', 'vardepsexclude', True) or ''
+        vardepsexclude += ' ' + (d.getVarFlag('IMAGE_CMD_' + realt, 'vardepsexclude', True) or '')
+        for var in vardepsexclude.split():
+            localdata.delVar(var)
         image_cmd = localdata.getVar("IMAGE_CMD", True)
         vardeps.add('IMAGE_CMD_' + realt)
         if image_cmd:

Signature · 1 line

2.9.3

Richard Purdie

2016-09-12 10:19 UTC #1 lore raw

Quoted 18 lines · Jonathan Liu

This fixes tashhash mismatch errors creating images when IMAGE_CMD references a variable whose value is always changing even though the variable is specified in IMAGE_CMD vardepsexclude.
Signed-off-by Jonathan Liu <[email protected]>
---
meta/classes/image.bbclass | 4 ++++
 1 file changed, 4 insertions(+)
meta/classes/image.bbclass +0 -0
index 8c8b8a2..f85ea86 100644
@@ -441,6 +441,10 @@ python () {
          localdata.delVar('DATETIME')
          localdata.delVar('TMPDIR')
+ vardepsexclude = d.getVarFlag('IMAGE_CMD', 'vardepsexclude',

Quoted 8 lines

True) or ''
+        vardepsexclude += ' ' + (d.getVarFlag('IMAGE_CMD_' + realt, 'vardepsexclude', True) or '')
+        for var in vardepsexclude.split():
+            localdata.delVar(var)
         image_cmd = localdata.getVar("IMAGE_CMD", True) vardeps.add('IMAGE_CMD_' + realt)
         if image_cmd:

I think this is complicating an already fragile situation and it worries me. Can you be more specific about which variable is causing problems here?

I'm starting to wonder if we should have an explicit flag set here for "problem" variables and we'd default to the values "DATETIME TMPDIR" which are already handled in the code. You'd then just need to extend that value. Using vardepsexclude seems to partially but not completely match?

Incidentally, I really want to complete some of the rework in this area so we get to "proper" functions and can stop playing games like this. I just haven't had the time to do that and this point in the release cycle isn't the right time to do that.

Cheers,

Richard

Jonathan Liu

author

2016-09-12 14:05 UTC #2 lore raw

Hi Richard,

Quoted 18 lines · Richard Purdie

Quoted 17 lines · Jonathan Liu
This fixes tashhash mismatch errors creating images when IMAGE_CMD references a variable whose value is always changing even though the variable is specified in IMAGE_CMD vardepsexclude.
Signed-off-by Jonathan Liu <[email protected]>
---
 meta/classes/image.bbclass | 4 ++++
 1 file changed, 4 insertions(+)
meta/classes/image.bbclass +0 -0
index 8c8b8a2..f85ea86 100644
@@ -441,6 +441,10 @@ python () {
         localdata.delVar('DATETIME')
         localdata.delVar('TMPDIR')
 

Quoted 13 lines

meta/classes/image.bbclass python () { · line 450
Quoted 9 lines
meta/classes/image.bbclass +1 -0
+        vardepsexclude = d.getVarFlag('IMAGE_CMD', 'vardepsexclude',
True) or ''
+ vardepsexclude += ' ' + (d.getVarFlag('IMAGE_CMD_' + realt,
'vardepsexclude', True) or '')
meta/classes/image.bbclass +2 -0
+        for var in vardepsexclude.split():
+            localdata.delVar(var)
         image_cmd = localdata.getVar("IMAGE_CMD", True)
         vardeps.add('IMAGE_CMD_' + realt)
         if image_cmd:
I think this is complicating an already fragile situation and it worries me. Can you be more specific about which variable is causing problems here?

See IMAGEDATESTAMP variable in:
https://git.yoctoproject.org/cgit/cgit.cgi/meta-raspberrypi/tree/classes/sdcard_image-rpi.bbclass
https://github.com/linux-sunxi/meta-sunxi/blob/master/classes/sdcard_image-sunxi.bbclass

You can test by changing to:
IMAGEDATESTAMP = "${@time.clock()}"

Quoted 7 lines

I'm starting to wonder if we should have an explicit flag set here for "problem" variables and we'd default to the values "DATETIME TMPDIR" which are already handled in the code. You'd then just need to extend that value. Using vardepsexclude seems to partially but not completely match?

Quoted 10 lines

Incidentally, I really want to complete some of the rework in this area so we get to "proper" functions and can stop playing games like this. I just haven't had the time to do that and this point in the release cycle isn't the right time to do that.
Cheers,
Richard

Regards,
Jonathan

Richard Purdie

2016-09-12 16:43 UTC #3 lore raw

Quoted 42 lines · Jonathan Liu

Hi Richard,
Quoted 23 lines · Richard Purdie
Quoted 21 lines · Jonathan Liu
This fixes tashhash mismatch errors creating images when IMAGE_CMD
references a variable whose value is always changing even though the
variable is specified in IMAGE_CMD vardepsexclude.
Signed-off-by Jonathan Liu <[email protected]>
---
meta/classes/image.bbclass | 4 ++++
 1 file changed, 4 insertions(+)
meta/classes/image.bbclass +0 -0
index 8c8b8a2..f85ea86 100644
@@ -441,6 +441,10 @@ python () {
          localdata.delVar('DATETIME')
          localdata.delVar('TMPDIR')
 
Quoted 15 lines
Quoted 11 lines
meta/classes/image.bbclass +1 -0
+        vardepsexclude = d.getVarFlag('IMAGE_CMD',
 'vardepsexclude',
True) or ''
+        vardepsexclude += ' ' + (d.getVarFlag('IMAGE_CMD_' + realt,
'vardepsexclude', True) or '')
+        for var in vardepsexclude.split():
+            localdata.delVar(var)
         image_cmd = localdata.getVar("IMAGE_CMD", True) vardeps.add('IMAGE_CMD_' + realt)
         if image_cmd:
I think this is complicating an already fragile situation and it worries me. Can you be more specific about which variable is causing
problems here?

Quoted 8 lines

See IMAGEDATESTAMP variable in:
https://git.yoctoproject.org/cgit/cgit.cgi/meta-raspberrypi/tree/clas
ses/sdcard_image-rpi.bbclass
https://github.com/linux-sunxi/meta-sunxi/blob/master/classes/sdcard_
image-sunxi.bbclass
You can test by changing to:
IMAGEDATESTAMP = "${@time.clock()}"

I appreciate this doesn't solve the underlying problem but is there a reason they can't just use ${DATE} there?

I mention this since DATE/TIME are handled by bitbake specially so that for example if your build rolls over midnight, it doesn't explode badly. If your images is generated pre midnight but them processed afterwards, I can see things working badly for that code...

Cheers,

Richard

Jonathan Liu

author

2016-09-13 06:53 UTC #4 lore raw

Hi Richard,

Quoted 49 lines · Richard Purdie

Quoted 48 lines · Jonathan Liu
Hi Richard,
Quoted 23 lines · Richard Purdie
Quoted 21 lines · Jonathan Liu
This fixes tashhash mismatch errors creating images when IMAGE_CMD
references a variable whose value is always changing even though the
variable is specified in IMAGE_CMD vardepsexclude.
Signed-off-by Jonathan Liu <[email protected]>
---
 meta/classes/image.bbclass | 4 ++++
 1 file changed, 4 insertions(+)
meta/classes/image.bbclass +0 -0
index 8c8b8a2..f85ea86 100644
@@ -441,6 +441,10 @@ python () {
         localdata.delVar('DATETIME')
         localdata.delVar('TMPDIR')
 
Quoted 15 lines
meta/classes/image.bbclass python () { · line 450
Quoted 11 lines
meta/classes/image.bbclass +1 -0
+        vardepsexclude = d.getVarFlag('IMAGE_CMD',
 'vardepsexclude',
True) or ''
+ vardepsexclude += ' ' + (d.getVarFlag('IMAGE_CMD_' +
realt,
'vardepsexclude', True) or '')
meta/classes/image.bbclass +2 -0
+        for var in vardepsexclude.split():
+            localdata.delVar(var)
         image_cmd = localdata.getVar("IMAGE_CMD", True)
         vardeps.add('IMAGE_CMD_' + realt)
         if image_cmd:
I think this is complicating an already fragile situation and it worries me. Can you be more specific about which variable is causing
problems here?
See IMAGEDATESTAMP variable in:
https://git.yoctoproject.org/cgit/cgit.cgi/meta-raspberrypi/tree/clas
ses/sdcard_image-rpi.bbclass
https://github.com/linux-sunxi/meta-sunxi/blob/master/classes/sdcard_
image-sunxi.bbclass

Quoted 5 lines

Quoted 2 lines
You can test by changing to:
IMAGEDATESTAMP = "${@time.clock()}"
I appreciate this doesn't solve the underlying problem but is there a reason they can't just use ${DATE} there?

I have submitted patches to meta-raspberrypi and meta-sunxi to remove IMAGEDATESTAMP since it is using IMAGE_NAME which already includes date and time.

Quoted 9 lines

I mention this since DATE/TIME are handled by bitbake specially so that for example if your build rolls over midnight, it doesn't explode badly. If your images is generated pre midnight but them processed afterwards, I can see things working badly for that code...
Cheers,
Richard

Regards,
Jonathan