Strange file names
I just did a pull on poky master and now have a lot of files
with strange names like these:
poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.0": }} linux-yocto-rt_3.0.bbappend poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.0": }} linux-yocto_3.0.bbappend poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend
Is this right? What are they all about?
Signature · 4 lines
------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------
Quoted 9 lines · Gary Thomas
I just did a pull on poky master and now have a lot of files with strange names like these:
poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.0": }} linux-yocto-rt_3.0.bbappend poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.0": }} linux-yocto_3.0.bbappend poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappendIs this right? What are they all about?
These are part of the new Yocto BSP Tools (yocto-bsp) and all the files under 'scripts/lib/bsp/substrate' are template files for the tool.
The template files can contain inline Python between {{ tags }} that gets used as logic when generating the BSP. The inline Python can also be used in the filne names themselves for that purpose. In this case, it's used to only include one of the .bbappend files depending on the value of the kernel_choice variable.
It may seem strange, but filenames are just text after all, and I wanted to avoid having separate 'meta-files' describing that same logic that would also have to be kept in sync, so decided it would be less error-prone to keep that information in the filenames themselves.
I'm in the process of writing all this up to put in the BSP manual, but not quite there yet...
Tom
Dear Tom Zanussi,
Quoted 6 lines · In message <1332859973.3854.15.camel@elmorro> you wrote:
The template files can contain inline Python between {{ tags }} that gets used as logic when generating the BSP. The inline Python can also be used in the filne names themselves for that purpose. In this case, it's used to only include one of the .bbappend files depending on the value of the
kernel_choicevariable.
Arghhh... This is totally broken. Do not do this.
Best regards,
Wolfgang Denk
Signature · 4 lines
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [email protected] Successful and fortunate crime is called virtue. - Seneca
Quoted 4 lines · Wolfgang Denk
Dear Tom Zanussi,
Quoted 1 line · In message <1332859973.3854.15.camel@elmorro> you wrote:
Quoted 8 lines
Quoted 5 lines
The template files can contain inline Python between {{ tags }} that gets used as logic when generating the BSP. The inline Python can also be used in the filne names themselves for that purpose. In this case, it's used to only include one of the .bbappend files depending on the value of the
kernel_choicevariable.Arghhh... This is totally broken. Do not do this.
Why do you say so - they're legal filenames and useful for this purpose?
Tom
Quoted 4 lines
Best regards,
Wolfgang Denk
Dear Tom Zanussi,
Quoted 4 lines · In message <1332870781.20140.12.camel@elmorro> you wrote:
Quoted 2 lines
Arghhh... This is totally broken. Do not do this.
Why do you say so - they're legal filenames and useful for this purpose?
Not anything that is legal (in the sense that it stretches the technical possibilities to the limit) actually makes sense.
Yes, Unix allows nearly arbitrary file names. Yes, it is technically possible to store the source code of a program or a script in a file name. The fact that nobody does this should make you think if this is a clever approach that nobody else could figure out, or if it is just plain ......
Please consider the mental welfare of the people who have to maintain that code after you, who have to type file names like "{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend".
Even standard commands like "ls" produce output that make my toe nails roll up:
$ ls scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/
files
{{ if kernel_choice == "linux-yocto_3.0": }} linux-yocto_3.0.bbappend
{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend
{{ if kernel_choice == "linux-yocto-rt_3.0": }} linux-yocto-rt_3.0.bbappend
{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend
kernel-list.noinstall
$PLEASE do not to this!
Best regards,
Wolfgang Denk
Signature · 5 lines
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [email protected] History is only a confused heap of facts. -- Philip Earl of Chesterfield
Quoted 8 lines · Wolfgang Denk
Dear Tom Zanussi,
Quoted 4 lines · In message <1332870781.20140.12.camel@elmorro> you wrote:
Quoted 2 lines
Arghhh... This is totally broken. Do not do this.
Why do you say so - they're legal filenames and useful for this purpose?
Quoted 9 lines
Not anything that is legal (in the sense that it stretches the technical possibilities to the limit) actually makes sense.
Yes, Unix allows nearly arbitrary file names. Yes, it is technically possible to store the source code of a program or a script in a file name. The fact that nobody does this should make you think if this is a clever approach that nobody else could figure out, or if it is just plain ......
Yeah, as I mentioned, the motivation was that it seemed the most straightforward thing to do, notwithstanding the ugliness of the file names.
Quoted 4 lines
Please consider the mental welfare of the people who have to maintain that code after you, who have to type file names like "{{ if
kernel_choice== "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend".
Believe me, I did consider it, since I was the one who had to time them in in the first place. ;-).
And I have to agree, it is a pain to edit them, mainly because of the need for quotes - actually I finally resorted to using Nautilus which made it much easier.
In the end, I balanced that against the fact that it would be very few people editing these files very rarely (once a release basically), and decided to do it anyway.
Quoted 12 lines
Even standard commands like "ls" produce output that make my toe nails roll up:
$ ls scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/ files {{ if kernel_choice == "linux-yocto_3.0": }} linux-yocto_3.0.bbappend {{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend {{ if kernel_choice == "linux-yocto-rt_3.0": }} linux-yocto-rt_3.0.bbappend {{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend kernel-list.noinstall $
Quoted 2 lines
PLEASE do not to this!
If everyone really hates this, I'm perfectly happy with getting rid of it - it's supposed to be a usability tool after all, and I definitely don't want to be doing the opposite...
Tom
Quoted 4 lines
Best regards,
Wolfgang Denk
Hi Tom,
Quoted 4 lines
It may seem strange, but filenames are just text after all, and I wanted to avoid having separate 'meta-files' describing that same logic that would also have to be kept in sync, so decided it would be less error-prone to keep that information in the filenames themselves.
It indeed is a possibility to encode "more than usual" into filenames, but it certainly will have other effects also. Now that the name of the file itself contains "business logic" that is important for the system, I wonder how this interacts with other parts of the (sometimes unwritten) expectations of the development environment. One thing which immediately comes to my mind is the interaction with the version control system.
Changing "an attribute" of such a file (which is coded in the inline python code) will essentially look like the deletion and addition of a file for version control. Of course our version control (git) will be clever enough to identify the file as being the same, but still we now have an operation which was previously not used to change "business logic", i.e. renaming files, doing just that. So for tracing the history of such an item I now need to trace the "rename history" additionally to the "content history".
So in contrast to the original idea of "keeping the information in one place", we now have two places, i.e. the content and the filename and the latter is really a very uncommon place for that.
I do have a severe bad feeling in my stomach about going into this (uncharted) territory and I do not know whether the potential problems that we (yet) do not see are worth it. So I would also like you to reconsider that strategy.
Thanks
Detlev
--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: [email protected]
Quoted 24 lines · Detlev Zundel
Hi Tom,
Quoted 4 lines
It may seem strange, but filenames are just text after all, and I wanted to avoid having separate 'meta-files' describing that same logic that would also have to be kept in sync, so decided it would be less error-prone to keep that information in the filenames themselves.
It indeed is a possibility to encode "more than usual" into filenames, but it certainly will have other effects also. Now that the name of the file itself contains "business logic" that is important for the system, I wonder how this interacts with other parts of the (sometimes unwritten) expectations of the development environment. One thing which immediately comes to my mind is the interaction with the version control system.
Changing "an attribute" of such a file (which is coded in the inline python code) will essentially look like the deletion and addition of a file for version control. Of course our version control (git) will be clever enough to identify the file as being the same, but still we now have an operation which was previously not used to change "business logic", i.e. renaming files, doing just that. So for tracing the history of such an item I now need to trace the "rename history" additionally to the "content history".
Quoted 9 lines
So in contrast to the original idea of "keeping the information in one place", we now have two places, i.e. the content and the filename and the latter is really a very uncommon place for that.
I do have a severe bad feeling in my stomach about going into this (uncharted) territory and I do not know whether the potential problems that we (yet) do not see are worth it. So I would also like you to reconsider that strategy.
Yes, I think it would make sense for various reasons to get rid of it if possible - it's not central to the mechanism, just convenient.
So the current uses of it are the following:
- having a particular filename match the machine name:
./powerpc/conf/machine/{{=machine}}.conf- having a particular filename conditionally included or not:
./i386/recipes-kernel/linux/{{ if kernel_choice =="linux-yocto_3.2": }} linux-yocto_3.2.bbappend
For those two types of cases, I think it should be possible to move that logic into the file itself using special filename and/or conditional filename directives.
The other usage is to have directory names match the machine name for instance:
./i386/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver_choice == "xserver_vesa": }} xorg.conf
In the case of directory names, since there's no file to put that directive into, and we can't use the directory name any longer, we'll have to again use some kind of special directory name directive and find some file to put it into, probably just another .noinstall file, which we already have a case of but that I wanted to get rid of. There may be a better way to do it, but we could keep those and do things that way in the worst case.
Tom
Quoted 12 lines
Thanks
Detlev--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: [email protected]_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 11 lines · Detlev Zundel
Hi Tom,
thanks for listening to my concerns. I have to admit though, that I wrote my mail because of my doubts about the usage of such unusal filenames, but I have yet to understand how you use the constructs.
Quoted 5 lines
Yes, I think it would make sense for various reasons to get rid of it if possible - it's not central to the mechanism, just convenient.
So the current uses of it are the following:
Quoted 8 lines
Quoted 3 lines
- having a particular filename match the machine name:
./powerpc/conf/machine/{{=machine}}.confCan you explain to me what this exactly does? Does that file match every machine name? If that is the case, then why do you need to match at all and not use a special file name?
I guess 'match' is the wrong word - basically what happens when processing the templates (and since the filenames themselves are part of the 'templates', those too), the {{=machine}} in the name is replaced with the name of the BSP. So in the final set of files, supposing you're creating a bsp named 'meta-foo', you end up with this file being named:
./powerpc/conf/machine/foo.conf
which is what's expected in the BSP layout.
Quoted 5 lines
Quoted 5 lines
- having a particular filename conditionally included or not:
./i386/recipes-kernel/linux/{{ if kernel_choice =="linux-yocto_3.2": }} linux-yocto_3.2.bbappend
Quoted 8 lines
Quoted 3 lines
For those two types of cases, I think it should be possible to move that logic into the file itself using special filename and/or conditional filename directives.
I'm not sure if I understand fully, but something like (shell-syntax) "include ${kernel_choice}.bbappend" would work - modulo ignoring errors if the file is not found of course.
So, similarly in this case, we have four files with similar names in this dir, but only one of them should be used, and which one depends on which kernel was selected. The 'template processing' uses the code in the filename to end up with that result. So your suggestion is partly fine in that we need some kind of 'include' to replace it, but it doesn't contain the logic we need to choose only that file. And it's just replacing it with something that looks like executable code anyway. But as I mentioned, I think this case also could me moved into the file itself.
Quoted 7 lines
Quoted 4 lines
The other usage is to have directory names match the machine name for instance:
./i386/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if
xserver_choice== "xserver_vesa": }} xorg.confI fail to understand this for the same reason than the first item ;)
This is actually a combination of the other two, replace the {{= with 'machine' but here it's a dirname, and conditionally select one of a set of files.
Again, in this case, for the file in this case, we can move the selection logic into the file, but for the dirname replacement we need another place to contain it, which we can use the aforementioned .noinstall files for, barring a better idea...
Tom
Quoted 7 lines
Cheers
Detlev--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: [email protected]
Hi Tom,
thanks for listening to my concerns. I have to admit though, that I wrote my mail because of my doubts about the usage of such unusal filenames, but I have yet to understand how you use the constructs.
Quoted 8 lines
Yes, I think it would make sense for various reasons to get rid of it if possible - it's not central to the mechanism, just convenient.
So the current uses of it are the following:
- having a particular filename match the machine name:
./powerpc/conf/machine/{{=machine}}.conf
Can you explain to me what this exactly does? Does that file match every machine name? If that is the case, then why do you need to match at all and not use a special file name?
Quoted 8 lines
- having a particular filename conditionally included or not:
./i386/recipes-kernel/linux/{{ if kernel_choice =="linux-yocto_3.2": }} linux-yocto_3.2.bbappend
For those two types of cases, I think it should be possible to move that logic into the file itself using special filename and/or conditional filename directives.
I'm not sure if I understand fully, but something like (shell-syntax) "include ${kernel_choice}.bbappend" would work - modulo ignoring errors if the file is not found of course.
Quoted 4 lines
The other usage is to have directory names match the machine name for instance:
./i386/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if
xserver_choice== "xserver_vesa": }} xorg.conf
I fail to understand this for the same reason than the first item ;)
Cheers
Detlev
--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: [email protected]