toaster: Add fake entry to Target_File for filesystem root
PATCH 1/1 toaster: Add fake entry to Target_File for filesystem root
The files-in-image.txt file is produced by bitbake after an image is created, listing all the files in the image. However, this list doesn't include the root directory ('/').
buildinfohelper.py then tries to construct the filesystem tree from this file, assuming that every directory apart from the root directory (which is special-cased) can be assigned a parent. But because the root directory isn't listed in files-in-image.txt, an object for the root directory is never created.
The direct subdirectories of the root ('./bin', './usr' etc.) then can't be assigned a parent directory, as the object representing the root directory doesn't exist. This results in a Target_File lookup error and causes the directory listing page to fail.
Fix this by creating a fake entry for the root directory in the Target_File table, so that the direct subdirectories of / can be assigned a parent. Note that it doesn't matter that the root is faked, as its properties are never shown in the directory structure tree.
[YOCTO #8280]
--- bitbake/lib/bb/ui/buildinfohelper.py | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +17 -14
@@ -353,26 +353,29 @@ class ORMWrapper(object): files = filedata['files'] syms = filedata['syms'] - # we insert directories, ordered by name depth + # always create the root directory as a special case; + # note that this is never displayed, so the owner, group, + # size, permission are irrelevant + tf_obj = Target_File.objects.create(target = target_obj, + path = '/', + size = 0, + owner = '', + group = '', + permission = '', + inodetype = Target_File.ITYPE_DIRECTORY) + tf_obj.save() + + # insert directories, ordered by name depth for d in sorted(dirs, key=lambda x:len(x[-1].split("/"))): (user, group, size) = d[1:4] permission = d[0][1:] path = d[4].lstrip(".") + + # we already created the root directory, so ignore any + # entry for it if len(path) == 0: - # we create the root directory as a special case - path = "/" - tf_obj = Target_File.objects.create( - target = target_obj, - path = path, - size = size, - inodetype = Target_File.ITYPE_DIRECTORY, - permission = permission, - owner = user, - group = group, - ) - tf_obj.directory = tf_obj - tf_obj.save() continue + parent_path = "/".join(path.split("/")[:len(path.split("/")) - 1]) if len(parent_path) == 0: parent_path = "/"
--
Elliot Smith
Software Engineer
Intel OTC
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
upstreamed.
ty,
b
Quoted 99 lines · Elliot Smith
The files-in-image.txt file is produced by bitbake after an image is created, listing all the files in the image. However, this list doesn't include the root directory ('/').
buildinfohelper.pythen tries to construct the filesystem tree from this file, assuming that every directory apart from the root directory (which is special-cased) can be assigned a parent. But because the root directory isn't listed in files-in-image.txt, an object for the root directory is never created.The direct subdirectories of the root ('./bin', './usr' etc.) then can't be assigned a parent directory, as the object representing the root directory doesn't exist. This results in a
Target_Filelookup error and causes the directory listing page to fail.Fix this by creating a fake entry for the root directory in the
Target_Filetable, so that the direct subdirectories of / can be assigned a parent. Note that it doesn't matter that the root is faked, as its properties are never shown in the directory structure tree.[YOCTO #8280]
Signed-off-by Elliot Smith <[email protected]>--- bitbake/lib/bb/ui/buildinfohelper.py | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-)bitbake/lib/bb/ui/buildinfohelper.py +17
-15@@ -353,26 +353,29 @@ class ORMWrapper(object): files = filedata['files'] syms = filedata['syms'] - # we insert directories, ordered by name depth + # always create the root directory as a special case; + # note that this is never displayed, so the owner, group, + # size, permission are irrelevant + tf_obj = Target_File.objects.create(target = target_obj, + path = '/', + size = 0, + owner = '', + group = '', + permission = '', + inodetype = Target_File.ITYPE_DIRECTORY) + tf_obj.save() + + # insert directories, ordered by name depth for d in sorted(dirs, key=lambda x:len(x[-1].split("/"))): (user, group, size) = d[1:4] permission = d[0][1:] path = d[4].lstrip(".") + + # we already created the root directory, so ignore any + # entry for it if len(path) == 0: - # we create the root directory as a special case - path = "/" - tf_obj = Target_File.objects.create( - target = target_obj, - path = path, - size = size, - inodetype = Target_File.ITYPE_DIRECTORY, - permission = permission, - owner = user, - group = group, - ) - tf_obj.directory = tf_obj - tf_obj.save() continue + parent_path = "/".join(path.split("/")[:len(path.split("/")) - 1]) if len(parent_path) == 0: parent_path = "/" --Elliot Smith
Software Engineer
Intel OTC---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster
Quoted 4 lines · Elliot Smith
The files-in-image.txt file is produced by bitbake after an image is created, listing all the files in the image. However, this list doesn't include the root directory ('/').
Did you try to look what caused this change? Bitbake used to include root dir into this file previously, right?
Quoted 13 lines
buildinfohelper.pythen tries to construct the filesystem tree from this file, assuming that every directory apart from the root directory (which is special-cased) can be assigned a parent. But because the root directory isn't listed in files-in-image.txt, an object for the root directory is never created.The direct subdirectories of the root ('./bin', './usr' etc.) then can't be assigned a parent directory, as the object representing the root directory doesn't exist. This results in a
Target_Filelookup error and causes the directory listing page to fail.
Quoted 6 lines
Fix this by creating a fake entry for the root directory in the
Target_Filetable, so that the direct subdirectories of / can be assigned a parent. Note that it doesn't matter that the root is faked, as its properties are never shown in the directory structure tree.
This looks like a workaround to me. It should be fixed in bitbake code if it's possible.
Regards,
Ed
Quoted 76 lines
[YOCTO #8280]
Signed-off-by Elliot Smith <[email protected]>--- bitbake/lib/bb/ui/buildinfohelper.py | 31 +++++++++++++++++-------------- 1 file changed, 17 insertions(+), 14 deletions(-)bitbake/lib/bb/ui/buildinfohelper.py +17
-15@@ -353,26 +353,29 @@ class ORMWrapper(object): files = filedata['files'] syms = filedata['syms'] - # we insert directories, ordered by name depth + # always create the root directory as a special case; + # note that this is never displayed, so the owner, group, + # size, permission are irrelevant + tf_obj = Target_File.objects.create(target = target_obj, + path = '/', + size = 0, + owner = '', + group = '', + permission = '', + inodetype = Target_File.ITYPE_DIRECTORY) + tf_obj.save() + + # insert directories, ordered by name depth for d in sorted(dirs, key=lambda x:len(x[-1].split("/"))): (user, group, size) = d[1:4] permission = d[0][1:] path = d[4].lstrip(".") + + # we already created the root directory, so ignore any + # entry for it if len(path) == 0: - # we create the root directory as a special case - path = "/" - tf_obj = Target_File.objects.create( - target = target_obj, - path = path, - size = size, - inodetype = Target_File.ITYPE_DIRECTORY, - permission = permission, - owner = user, - group = group, - ) - tf_obj.directory = tf_obj - tf_obj.save() continue + parent_path = "/".join(path.split("/")[:len(path.split("/")) - 1]) if len(parent_path) == 0: parent_path = "/" --Elliot Smith
Software Engineer
Intel OTC---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Signature · 4 lines
_______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toaster
Signature · 3 lines
-- Regards, Ed
Quoted 8 lines · Ed Bartosh
Quoted 4 lines · Elliot Smith
The files-in-image.txt file is produced by bitbake after an image is created, listing all the files in the image. However, this list doesn't include the root directory ('/').
Did you try to look what caused this change? Bitbake used to include root dir into this file previously, right?
I'm not sure if it ever did include this file. I had a brief look at how buildhistory_analysis.py does its work (git diffs if I read it correctly), but didn't dig right into it.
Quoted 13 lines
Quoted 13 lines
buildinfohelper.pythen tries to construct the filesystem tree from this file, assuming that every directory apart from the root directory (which is special-cased) can be assigned a parent. But because the root directory isn't listed in files-in-image.txt, an object for the root directory is never created.The direct subdirectories of the root ('./bin', './usr' etc.) then can't be assigned a parent directory, as the object representing the root directory doesn't exist. This results in a
Target_Filelookup error and causes the directory listing page to fail.
Quoted 9 lines
Quoted 6 lines
Fix this by creating a fake entry for the root directory in the
Target_Filetable, so that the direct subdirectories of / can be assigned a parent. Note that it doesn't matter that the root is faked, as its properties are never shown in the directory structure tree.This looks like a workaround to me. It should be fixed in bitbake code if it's possible.
I discussed this with Brian, and the decision was that a work-around was
sufficient for now. (See the options I suggested in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8280#c9.)
I can investigate this some more and/or raise a bug, if we think that's worthwhile.
Elliot
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre
yes, please raise the buildhistory bug with bitbake but for now, I'm ok with this workaround, especially since you did the sensible thing of checking to see if it exists before creating your entry i.e. this code will still work even after/if they fix the buildhistory bug. -b
Quoted 59 lines · Elliot
Quoted 8 lines · Ed Bartosh
Quoted 4 lines · Elliot Smith
The files-in-image.txt file is produced by bitbake after an image is created, listing all the files in the image. However, this list doesn't include the root directory ('/').
Did you try to look what caused this change? Bitbake used to include root dir into this file previously, right?
I'm not sure if it ever did include this file. I had a brief look at how
buildhistory_analysis.pydoes its work (git diffs if I read it correctly), but didn't dig right into it.Quoted 14 lines
Quoted 14 lines
buildinfohelper.pythen tries to construct the filesystem tree from this file, assuming that every directory apart from the root directory (which is special-cased) can be assigned a parent. But because the root directory isn't listed in files-in-image.txt, an object for the root directory is never created.The direct subdirectories of the root ('./bin', './usr' etc.) then can't be assigned a parent directory, as the object representing the root directory doesn't exist. This results in a
Target_Filelookup error and causes the directory listing page to fail.Quoted 8 lines
Quoted 6 lines
Fix this by creating a fake entry for the root directory in the
Target_Filetable, so that the direct subdirectories of / can be assigned a parent. Note that it doesn't matter that the root is faked, as its properties are never shown in the directory structure tree.This looks like a workaround to me. It should be fixed in bitbake code if it's possible.
I discussed this with Brian, and the decision was that a work-around was
sufficient for now. (See the options I suggested in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8280#c9.)I can investigate this some more and/or raise a bug, if we think that's worthwhile.
Elliot
--
Elliot Smith
Software Engineer
Intel Open Source Technology Centre--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster
Quoted 5 lines · Brian Avery
yes, please raise the buildhistory bug with bitbake but for now, I'm ok with this workaround, especially since you did the sensible thing of checking to see if it exists before creating your entry i.e. this code will still work even after/if they fix the buildhistory bug.
I raised a bug for the buildhistory issue:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8370
Elliot
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre