Skip to contentSkip to conversation

classes/image: Use xargs to set file timestamps

PATCH In review Joshua Watt 3 participants 4 messages May 5, 2021 – May 6, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 classes/image: Use xargs to set file timestamps

Instead of having find directly invoke touch for each file in the root file system, pass a list to xargs for batching. This significantly reduces the number of times the touch program is invoked and speeds up the do_image task time:

    PKG           TASK      ABSDIFF  RELDIFF  CPUTIME1 -> CPUTIME2
    my-image      do_image   -45.3s   -94.2%     48.1s -> 2.8s

    Cumulative cputime:
      -44.3s    -92.3%    00:48.1 (48.1s) -> 00:03.7 (3.7s)
---
 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/image.bbclass +1 -1
index 013455f492..353cc67175 100644
@@ -657,7 +657,7 @@ reproducible_final_image_task () {
         fi
         # Set mtime of all files to a reproducible value
         bbnote "reproducible_final_image_task: mtime set to $REPRODUCIBLE_TIMESTAMP_ROOTFS"
-        find  ${IMAGE_ROOTFS} -exec touch -h  --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS {} \;
+        find  ${IMAGE_ROOTFS} -print0 | xargs -0 touch -h  --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS
     fi
 }
 
Signature · 1 line
2.31.1
#1 lore raw

Can we backport this to hardknott and dunfell (and gatesgarth if it's still active)?

Quoted 29 lines · Joshua Watt

Instead of having find directly invoke touch for each file in the root file system, pass a list to xargs for batching. This significantly reduces the number of times the touch program is invoked and speeds up the do_image task time:

     PKG           TASK      ABSDIFF  RELDIFF  CPUTIME1 -> CPUTIME2
     my-image      do_image   -45.3s   -94.2%     48.1s -> 2.8s

     Cumulative cputime:
       -44.3s    -92.3%    00:48.1 (48.1s) -> 00:03.7 (3.7s)
---
  meta/classes/image.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/image.bbclass +1 -1
index 013455f492..353cc67175 100644
@@ -657,7 +657,7 @@ reproducible_final_image_task () {
          fi
          # Set mtime of all files to a reproducible value
          bbnote "reproducible_final_image_task: mtime set to $REPRODUCIBLE_TIMESTAMP_ROOTFS"
-        find  ${IMAGE_ROOTFS} -exec touch -h  --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS {} \;
+        find  ${IMAGE_ROOTFS} -print0 | xargs -0 touch -h  --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS
      fi
  }
 
#2 lore raw
Quoted 2 lines · Joshua Watt

Can we backport this to hardknott and dunfell (and gatesgarth if it's still active)?

Yes indeed, please to dunfell!

-Mikko, who also marked the patch but didn't have time to put it to test on dunfell..

Quoted 33 lines · Joshua Watt
Quoted 28 lines

Instead of having find directly invoke touch for each file in the root file system, pass a list to xargs for batching. This significantly reduces the number of times the touch program is invoked and speeds up the do_image task time:

     PKG           TASK      ABSDIFF  RELDIFF  CPUTIME1 -> CPUTIME2
     my-image      do_image   -45.3s   -94.2%     48.1s -> 2.8s

     Cumulative cputime:
       -44.3s    -92.3%    00:48.1 (48.1s) -> 00:03.7 (3.7s)
---
  meta/classes/image.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/image.bbclass +1 -1
index 013455f492..353cc67175 100644
@@ -657,7 +657,7 @@ reproducible_final_image_task () {
          fi
          # Set mtime of all files to a reproducible value
          bbnote "reproducible_final_image_task: mtime set to $REPRODUCIBLE_TIMESTAMP_ROOTFS"
-        find  ${IMAGE_ROOTFS} -exec touch -h  --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS {} \;
+        find  ${IMAGE_ROOTFS} -print0 | xargs -0 touch -h  --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS
      fi
  }
#3 lore raw
In reply to Joshua Watt
Quoted 3 lines · Joshua Watt

Can we backport this to hardknott and dunfell (and gatesgarth if it's still active)?

Yes, will backport to dunfell!

Steve

Quoted 30 lines · Joshua Watt
Quoted 29 lines

Instead of having find directly invoke touch for each file in the root file system, pass a list to xargs for batching. This significantly reduces the number of times the touch program is invoked and speeds up the do_image task time:

     PKG           TASK      ABSDIFF  RELDIFF  CPUTIME1 -> CPUTIME2
     my-image      do_image   -45.3s   -94.2%     48.1s -> 2.8s

     Cumulative cputime:
       -44.3s    -92.3%    00:48.1 (48.1s) -> 00:03.7 (3.7s)
---
  meta/classes/image.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/image.bbclass +1 -1
index 013455f492..353cc67175 100644
@@ -657,7 +657,7 @@ reproducible_final_image_task () {
          fi
          # Set mtime of all files to a reproducible value
          bbnote "reproducible_final_image_task: mtime set to $REPRODUCIBLE_TIMESTAMP_ROOTFS"
-        find  ${IMAGE_ROOTFS} -exec touch -h  --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS {} \;
+        find  ${IMAGE_ROOTFS} -print0 | xargs -0 touch -h  --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS
      fi
  }