Skip to contentSkip to conversation

sstate.bbclass: update the timestamps after install

PATCH In review Robert Yang 3 participants 9 messages Sep 4, 2014 – Sep 25, 2014

lore
Share & tools
lore
#0 lore raw

cover sstate.bbclass: update the timestamps after install

The following changes since commit 4c0a9ccbad2889b27b4b1d2ab91215a4bdcca3ce:

python-numpy: Fix build for mips64 (2014-09-01 18:00:32 +0100)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib rbt/ss
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=rbt/ss

Robert Yang (1):
sstate.bbclass: update the timestamps after install

 meta/classes/sstate.bbclass |    1 +
 1 file changed, 1 insertion(+)
Signature · 1 line
1.7.9.5
#1 lore raw

PATCH 1/1 sstate.bbclass: update the timestamps after install

Update the sstate file's timestamps after it is installed, it will be very useful for removing the old sstate file, especially, it's not easy to remove when use the shared SSTATE_DIR, we can easily remove them with this change, for example:

$ find state-cache -type f -ctime +10 -exec rm -f {} \;

Will remove the sstate file which isn't used by recent 10 days.

We can use the -atime, but it is not always available, for example, when mounted with "-o noatime".

The touch is a very light weight action, and the scripts/sstate-cache-management.sh also requires this.

---
 meta/classes/sstate.bbclass |    1 +
 1 file changed, 1 insertion(+)
meta/classes/sstate.bbclass +1 -0
index ead829e..885912d 100644
@@ -618,6 +618,7 @@ sstate_unpack_package () {
 	mkdir -p ${SSTATE_INSTDIR}
 	cd ${SSTATE_INSTDIR}
 	tar -xmvzf ${SSTATE_PKG}
+	touch --no-dereference ${SSTATE_PKG}
 }
 
 BB_HASHCHECK_FUNCTION = "sstate_checkhashes"
Signature · 1 line
1.7.9.5
#2 lore raw
Quoted 25 lines · Robert Yang

Update the sstate file's timestamps after it is installed, it will be very useful for removing the old sstate file, especially, it's not easy to remove when use the shared SSTATE_DIR, we can easily remove them with this change, for example:

$ find state-cache -type f -ctime +10 -exec rm -f {} \;

Will remove the sstate file which isn't used by recent 10 days.

We can use the -atime, but it is not always available, for example, when mounted with "-o noatime".

The touch is a very light weight action, and the scripts/sstate-cache-management.sh also requires this.

---
 meta/classes/sstate.bbclass |    1 +
 1 file changed, 1 insertion(+)
meta/classes/sstate.bbclass +0 -0
index ead829e..885912d 100644

meta/classes/sstate.bbclass sstate_unpack_package () { · line 624

Quoted 7 lines
meta/classes/sstate.bbclass +1 -0
@@ -618,6 +618,7 @@ sstate_unpack_package () {
 	mkdir -p ${SSTATE_INSTDIR}
 	cd ${SSTATE_INSTDIR}
 	tar -xmvzf ${SSTATE_PKG}
+	touch --no-dereference ${SSTATE_PKG}
 }
 
 BB_HASHCHECK_FUNCTION = "sstate_checkhashes"

At the very least we need to consider read only files here...

Cheers,

Richard

#3 lore raw
Quoted 32 lines · Richard Purdie
Quoted 31 lines · Robert Yang

Update the sstate file's timestamps after it is installed, it will be very useful for removing the old sstate file, especially, it's not easy to remove when use the shared SSTATE_DIR, we can easily remove them with this change, for example:

$ find state-cache -type f -ctime +10 -exec rm -f {} \;

Will remove the sstate file which isn't used by recent 10 days.

We can use the -atime, but it is not always available, for example, when mounted with "-o noatime".

The touch is a very light weight action, and the scripts/sstate-cache-management.sh also requires this.

---
  meta/classes/sstate.bbclass |    1 +
  1 file changed, 1 insertion(+)
meta/classes/sstate.bbclass +1 -0
index ead829e..885912d 100644
@@ -618,6 +618,7 @@ sstate_unpack_package () {
  	mkdir -p ${SSTATE_INSTDIR}
  	cd ${SSTATE_INSTDIR}
  	tar -xmvzf ${SSTATE_PKG}
+	touch --no-dereference ${SSTATE_PKG}
  }
 
Quoted 3 lines

meta/classes/sstate.bbclass sstate_unpack_package () { · line 624

Quoted 1 line
meta/classes/sstate.bbclass +0 -0
  BB_HASHCHECK_FUNCTION = "sstate_checkhashes"

At the very least we need to consider read only files here...

Hello, did you mean the SSTATE_MIRRORS ? I thought that I had considered it since I used the "touch --no-dereference", make a clear check is reasonable if I understand correctly, so updated the code in the repo: (The --no-dereference is not need any more since the "test -w" follows symlink).

meta/classes/sstate.bbclass +2 -0
index ead829e..7cb43b1 100644
@@ -618,6 +618,8 @@ sstate_unpack_package () {
        mkdir -p ${SSTATE_INSTDIR}
        cd ${SSTATE_INSTDIR}
        tar -xmvzf ${SSTATE_PKG}
+       # Use "! test -w ||" to return true for read only files
+       [ ! test -w ${SSTATE_PKG} ] || touch ${SSTATE_PKG}
 }
 

// Robert

Quoted 7 lines

Cheers,

Richard

#4 lore raw
Quoted 38 lines · Robert Yang
Quoted 32 lines · Richard Purdie
Quoted 31 lines · Robert Yang

Update the sstate file's timestamps after it is installed, it will be very useful for removing the old sstate file, especially, it's not easy to remove when use the shared SSTATE_DIR, we can easily remove them with this change, for example:

$ find state-cache -type f -ctime +10 -exec rm -f {} \;

Will remove the sstate file which isn't used by recent 10 days.

We can use the -atime, but it is not always available, for example, when mounted with "-o noatime".

The touch is a very light weight action, and the scripts/sstate-cache-management.sh also requires this.

---
  meta/classes/sstate.bbclass |    1 +
  1 file changed, 1 insertion(+)
meta/classes/sstate.bbclass +1 -0
index ead829e..885912d 100644
@@ -618,6 +618,7 @@ sstate_unpack_package () {
  	mkdir -p ${SSTATE_INSTDIR}
  	cd ${SSTATE_INSTDIR}
  	tar -xmvzf ${SSTATE_PKG}
+	touch --no-dereference ${SSTATE_PKG}
  }
 
Quoted 3 lines

meta/classes/sstate.bbclass sstate_unpack_package () { · line 624

Quoted 1 line
meta/classes/sstate.bbclass +0 -0
  BB_HASHCHECK_FUNCTION = "sstate_checkhashes"

At the very least we need to consider read only files here...

Quoted 5 lines

Hello, did you mean the SSTATE_MIRRORS ? I thought that I had considered it since I used the "touch --no-dereference", make a clear check is reasonable if I understand correctly, so updated the code in the repo: (The --no-dereference is not need any more since the "test -w" follows symlink).

We need the --no-dereferece but this could also be a readonly directory or set of readonly files (think an ISO shipped with WR Linux or locked sstate cache).

Cheers,

Richard

#5 lore raw
Quoted 39 lines · Richard Purdie
Quoted 38 lines · Robert Yang
Quoted 35 lines · Richard Purdie
Quoted 25 lines · Robert Yang

Update the sstate file's timestamps after it is installed, it will be very useful for removing the old sstate file, especially, it's not easy to remove when use the shared SSTATE_DIR, we can easily remove them with this change, for example:

$ find state-cache -type f -ctime +10 -exec rm -f {} \;

Will remove the sstate file which isn't used by recent 10 days.

We can use the -atime, but it is not always available, for example, when mounted with "-o noatime".

The touch is a very light weight action, and the scripts/sstate-cache-management.sh also requires this.

---
   meta/classes/sstate.bbclass |    1 +
   1 file changed, 1 insertion(+)
meta/classes/sstate.bbclass +0 -0
index ead829e..885912d 100644

meta/classes/sstate.bbclass sstate_unpack_package () { · line 624

Quoted 7 lines
meta/classes/sstate.bbclass +1 -0
@@ -618,6 +618,7 @@ sstate_unpack_package () {
   	mkdir -p ${SSTATE_INSTDIR}
   	cd ${SSTATE_INSTDIR}
   	tar -xmvzf ${SSTATE_PKG}
+	touch --no-dereference ${SSTATE_PKG}
   }
 
   BB_HASHCHECK_FUNCTION = "sstate_checkhashes"

At the very least we need to consider read only files here...

Quoted 7 lines
Quoted 5 lines

Hello, did you mean the SSTATE_MIRRORS ? I thought that I had considered it since I used the "touch --no-dereference", make a clear check is reasonable if I understand correctly, so updated the code in the repo: (The --no-dereference is not need any more since the "test -w" follows symlink).

We need the --no-dereferece but this could also be a readonly directory

Under more thoughts, yes, we need the --no-dereferece otherwise the touch may update the file in the SSTATE_MIRRORS, and it's safe to touch a symlink itself (though "test -w" follows symlink) since the permissions of symbolic links are never used.

For readonly_dir/ssfile, we can touch the ssfile if "test -w ssfile", it doesn't care about the dir's write permission. Or do we have to check the permission of "dirname ${SSTATE_PKG}" and ${SSTATE_DIR} ? I don't think that we need unless it causes errors.

For the ISO, the "test -w {SSTATE_PKG}" would fail so that no touch, here is the updated code, also in the repo:

meta/classes/sstate.bbclass +2 -0
index ead829e..78766c0 100644
@@ -618,6 +618,8 @@ sstate_unpack_package () {
        mkdir -p ${SSTATE_INSTDIR}
        cd ${SSTATE_INSTDIR}
        tar -xmvzf ${SSTATE_PKG}
+       # Use "! -w ||" to return true for read only files
+       [ ! -w ${SSTATE_PKG} ] || touch ${SSTATE_PKG}
 }
 

// Robert

Quoted 9 lines

or set of readonly files (think an ISO shipped with WR Linux or locked sstate cache).

Cheers,

Richard

#6 lore raw
Quoted 48 lines · Robert Yang
Quoted 45 lines · Richard Purdie
Quoted 43 lines · Robert Yang
Quoted 32 lines · Richard Purdie
Quoted 31 lines · Robert Yang

Update the sstate file's timestamps after it is installed, it will be very useful for removing the old sstate file, especially, it's not easy to remove when use the shared SSTATE_DIR, we can easily remove them with this change, for example:

$ find state-cache -type f -ctime +10 -exec rm -f {} \;

Will remove the sstate file which isn't used by recent 10 days.

We can use the -atime, but it is not always available, for example, when mounted with "-o noatime".

The touch is a very light weight action, and the scripts/sstate-cache-management.sh also requires this.

---
   meta/classes/sstate.bbclass |    1 +
   1 file changed, 1 insertion(+)
meta/classes/sstate.bbclass +1 -0
index ead829e..885912d 100644
@@ -618,6 +618,7 @@ sstate_unpack_package () {
       mkdir -p ${SSTATE_INSTDIR}
       cd ${SSTATE_INSTDIR}
       tar -xmvzf ${SSTATE_PKG}
+    touch --no-dereference ${SSTATE_PKG}
   }
 
Quoted 3 lines

meta/classes/sstate.bbclass sstate_unpack_package () { · line 624

Quoted 1 line
meta/classes/sstate.bbclass +0 -0
   BB_HASHCHECK_FUNCTION = "sstate_checkhashes"

At the very least we need to consider read only files here...

Hello, did you mean the SSTATE_MIRRORS ? I thought that I had considered it since I used the "touch --no-dereference", make a clear check is reasonable if I understand correctly, so updated the code in the repo: (The --no-dereference is not need any more since the "test -w" follows symlink).

Quoted 3 lines
Quoted 1 line

We need the --no-dereferece but this could also be a readonly directory

Under more thoughts, yes, we need the --no-dereferece otherwise the

s/Under/After/

// Robert

Quoted 35 lines

touch may update the file in the SSTATE_MIRRORS, and it's safe to touch a symlink itself (though "test -w" follows symlink) since the permissions of symbolic links are never used.

For readonly_dir/ssfile, we can touch the ssfile if "test -w ssfile", it doesn't care about the dir's write permission. Or do we have to check the permission of "dirname ${SSTATE_PKG}" and ${SSTATE_DIR} ? I don't think that we need unless it causes errors.

For the ISO, the "test -w {SSTATE_PKG}" would fail so that no touch, here is the updated code, also in the repo:

meta/classes/sstate.bbclass +2 -0
index ead829e..78766c0 100644
@@ -618,6 +618,8 @@ sstate_unpack_package () {
         mkdir -p ${SSTATE_INSTDIR}
         cd ${SSTATE_INSTDIR}
         tar -xmvzf ${SSTATE_PKG}
+       # Use "! -w ||" to return true for read only files
+       [ ! -w ${SSTATE_PKG} ] || touch ${SSTATE_PKG}
  }
 
 // Robert
Quoted 10 lines

or set of readonly files (think an ISO shipped with WR Linux or locked sstate cache).

Cheers,

Richard

#7 lore raw
In reply to Robert Yang

Hi Robert,

Quoted 5 lines · Robert Yang

Under more thoughts, yes, we need the --no-dereferece otherwise the touch may update the file in the SSTATE_MIRRORS, and it's safe to touch a symlink itself (though "test -w" follows symlink) since the permissions of symbolic links are never used.

Quoted 7 lines

For readonly_dir/ssfile, we can touch the ssfile if "test -w ssfile", it doesn't care about the dir's write permission. Or do we have to check the permission of "dirname ${SSTATE_PKG}" and ${SSTATE_DIR} ? I don't think that we need unless it causes errors.

For the ISO, the "test -w {SSTATE_PKG}" would fail so that no touch, here is the updated code, also in the repo:

Can you update this patch with --no-dereference please, or have I misunderstood the conversation?

Ross

#8 lore raw
Quoted 13 lines · Ross

Hi Robert,

Quoted 10 lines · Robert Yang

Under more thoughts, yes, we need the --no-dereferece otherwise the touch may update the file in the SSTATE_MIRRORS, and it's safe to touch a symlink itself (though "test -w" follows symlink) since the permissions of symbolic links are never used.

For readonly_dir/ssfile, we can touch the ssfile if "test -w ssfile", it doesn't care about the dir's write permission. Or do we have to check the permission of "dirname ${SSTATE_PKG}" and ${SSTATE_DIR} ? I don't think that we need unless it causes errors.

Quoted 5 lines
Quoted 2 lines

For the ISO, the "test -w {SSTATE_PKG}" would fail so that no touch, here is the updated code, also in the repo:

Can you update this patch with --no-dereference please, or have I misunderstood the conversation?

Thanks, updated:

git://git.openembedded.org/openembedded-core-contrib rbt/ss

meta/classes/sstate.bbclass +2 -0
index 4057c8c..552ff8d 100644
@@ -623,6 +623,8 @@ sstate_unpack_package () {
        mkdir -p ${SSTATE_INSTDIR}
        cd ${SSTATE_INSTDIR}
        tar -xmvzf ${SSTATE_PKG}
+       # Use "! -w ||" to return true for read only files
+       [ ! -w ${SSTATE_PKG} ] || touch --no-dereference ${SSTATE_PKG}
 }
 

// Robert

Quoted 4 lines

Ross