bitbake: cooker: Ensure bbappends are found in stable order
Quoted 34 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Wes Lindauer
Sent: den 28 augusti 2019 23:38
To: [email protected]
Subject: [OE-core] [PATCH] bitbake: cooker: Ensure bbappends are found in stable orderThanks to wildcards in bbappend filenames, it's possible to have multiple bbappends that apply to the same recipe in the same directory. In order to get sstate hits between different workspaces, we want to apply those bbappend files in a consistent order. Since
readdir()returns files in a non-deterministic order between workspaces (based on inode number and/or time of creation), we'll need to sort its result in order to have any consistency.Signed-off-by Wes Lindauer <[email protected]>--- bitbake/lib/bb/cooker.py | 1 + 1 file changed, 1 insertion(+)bitbake/lib/bb/cooker.py +1
-1@@ -1869,6 +1869,7 @@ class CookerCollectFiles(object): (bbappend, filename) = b if (bbappend == f) or ('%' in bbappend and bbappend.startswith(f[:bbappend.index('%')])): filelist.append(filename) + filelist.sort() return filelist def collection_priorities(self, pkgfns, d): --
Quoted 1 line
2.14.5
I don't think this was a good idea at all. Before, bbappends were applied in the priority order of the layers, but this change throws the sorting based on priority out the window and might cause bbappend files for higher layers to be applied before lower layers' bbappend files.
Please revert this change, and rethink how to maintain a stable order for bbappend files from the same directory without affecting bbappend files from different layers.
//Peter
Quoted 41 lines · Peter Kjellerstedt
Quoted 34 lines
-----Original Message-----
From: [email protected] <openembedded- [email protected]> On Behalf Of Wes Lindauer
Sent: den 28 augusti 2019 23:38
To: [email protected]
Subject: [OE-core] [PATCH] bitbake: cooker: Ensure bbappends are found in stable orderThanks to wildcards in bbappend filenames, it's possible to have multiple bbappends that apply to the same recipe in the same directory. In order to get sstate hits between different workspaces, we want to apply those bbappend files in a consistent order. Since
readdir()returns files in a non-deterministic order between workspaces (based on inode number and/or time of creation), we'll need to sort its result in order to have any consistency.Signed-off-by Wes Lindauer <[email protected]>--- bitbake/lib/bb/cooker.py | 1 + 1 file changed, 1 insertion(+)bitbake/lib/bb/cooker.py +1
-1@@ -1869,6 +1869,7 @@ class CookerCollectFiles(object): (bbappend, filename) = b if (bbappend == f) or ('%' in bbappend and bbappend.startswith(f[:bbappend.index('%')])): filelist.append(filename) + filelist.sort() return filelist def collection_priorities(self, pkgfns, d): --Quoted 1 line
2.14.5
I don't think this was a good idea at all. Before, bbappends were applied in the priority order of the layers, but this change throws the sorting based on priority out the window and might cause bbappend files for higher layers to be applied before lower layers' bbappend files.
Quoted 3 lines
Please revert this change, and rethink how to maintain a stable order for bbappend files from the same directory without affecting bbappend files from different layers.
I had assumed that the sorting was only within each layer's bbappends. If it's global, you are absolutely right that priority order matters.
So maybe:
Within a layer (multiple .bbappends) 'sort' (alphabetic)
Between layers first by priority, then by order in the bblayers (for two layers with the same priority)?
--Mark
Quoted 2 lines
//Peter