Skip to contentSkip to conversation

Strange file names

DISCUSSION Gary Thomas 4 participants 10 messages Mar 27, 2012 – Mar 29, 2012 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

I just did a pull on poky master and now have a lot of files
with strange names like these:
poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.0": }} linux-yocto-rt_3.0.bbappend poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.0": }} linux-yocto_3.0.bbappend poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend

Is this right? What are they all about?

Signature · 4 lines
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------
#1 lore raw
Quoted 9 lines · Gary Thomas

I just did a pull on poky master and now have a lot of files with strange names like these:

   poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.0": }} linux-yocto-rt_3.0.bbappend
   poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend
   poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.0": }} linux-yocto_3.0.bbappend
   poky/scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend

Is this right? What are they all about?

These are part of the new Yocto BSP Tools (yocto-bsp) and all the files under 'scripts/lib/bsp/substrate' are template files for the tool.

The template files can contain inline Python between {{ tags }} that gets used as logic when generating the BSP. The inline Python can also be used in the filne names themselves for that purpose. In this case, it's used to only include one of the .bbappend files depending on the value of the kernel_choice variable.

It may seem strange, but filenames are just text after all, and I wanted to avoid having separate 'meta-files' describing that same logic that would also have to be kept in sync, so decided it would be less error-prone to keep that information in the filenames themselves.

I'm in the process of writing all this up to put in the BSP manual, but not quite there yet...

Tom

#2 lore raw

Dear Tom Zanussi,

Quoted 6 lines · In message <1332859973.3854.15.camel@elmorro> you wrote:

The template files can contain inline Python between {{ tags }} that gets used as logic when generating the BSP. The inline Python can also be used in the filne names themselves for that purpose. In this case, it's used to only include one of the .bbappend files depending on the value of the kernel_choice variable.

Arghhh... This is totally broken. Do not do this.

Best regards,

Wolfgang Denk

Signature · 4 lines
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [email protected]
Successful and fortunate crime is called virtue.             - Seneca
#3 lore raw
Quoted 4 lines · Wolfgang Denk

Dear Tom Zanussi,

Quoted 1 line · In message <1332859973.3854.15.camel@elmorro> you wrote:
Quoted 8 lines
Quoted 5 lines

The template files can contain inline Python between {{ tags }} that gets used as logic when generating the BSP. The inline Python can also be used in the filne names themselves for that purpose. In this case, it's used to only include one of the .bbappend files depending on the value of the kernel_choice variable.

Arghhh... This is totally broken. Do not do this.

Why do you say so - they're legal filenames and useful for this purpose?

Tom

Quoted 4 lines

Best regards,

Wolfgang Denk

#4 lore raw

Dear Tom Zanussi,

Quoted 4 lines · In message <1332870781.20140.12.camel@elmorro> you wrote:
Quoted 2 lines

Arghhh... This is totally broken. Do not do this.

Why do you say so - they're legal filenames and useful for this purpose?

Not anything that is legal (in the sense that it stretches the technical possibilities to the limit) actually makes sense.

Yes, Unix allows nearly arbitrary file names. Yes, it is technically possible to store the source code of a program or a script in a file name. The fact that nobody does this should make you think if this is a clever approach that nobody else could figure out, or if it is just plain ......

Please consider the mental welfare of the people who have to maintain that code after you, who have to type file names like "{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend".

Even standard commands like "ls" produce output that make my toe nails roll up:

$ ls scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/
files
{{ if kernel_choice == "linux-yocto_3.0": }} linux-yocto_3.0.bbappend
{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend
{{ if kernel_choice == "linux-yocto-rt_3.0": }} linux-yocto-rt_3.0.bbappend
{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend
kernel-list.noinstall
$

PLEASE do not to this!

Best regards,

Wolfgang Denk

Signature · 5 lines
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: [email protected]
History is only a confused heap of facts.
                                       -- Philip Earl of Chesterfield
#5 lore raw
Quoted 8 lines · Wolfgang Denk

Dear Tom Zanussi,

Quoted 4 lines · In message <1332870781.20140.12.camel@elmorro> you wrote:
Quoted 2 lines

Arghhh... This is totally broken. Do not do this.

Why do you say so - they're legal filenames and useful for this purpose?

Quoted 9 lines

Not anything that is legal (in the sense that it stretches the technical possibilities to the limit) actually makes sense.

Yes, Unix allows nearly arbitrary file names. Yes, it is technically possible to store the source code of a program or a script in a file name. The fact that nobody does this should make you think if this is a clever approach that nobody else could figure out, or if it is just plain ......

Yeah, as I mentioned, the motivation was that it seemed the most straightforward thing to do, notwithstanding the ugliness of the file names.

Quoted 4 lines

Please consider the mental welfare of the people who have to maintain that code after you, who have to type file names like "{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend".

Believe me, I did consider it, since I was the one who had to time them in in the first place. ;-).

And I have to agree, it is a pain to edit them, mainly because of the need for quotes - actually I finally resorted to using Nautilus which made it much easier.

In the end, I balanced that against the fact that it would be very few people editing these files very rarely (once a release basically), and decided to do it anyway.

Quoted 12 lines

Even standard commands like "ls" produce output that make my toe nails roll up:

$ ls scripts/lib/bsp/substrate/target/arch/x86_64/recipes-kernel/linux/
files
{{ if kernel_choice == "linux-yocto_3.0": }} linux-yocto_3.0.bbappend
{{ if kernel_choice == "linux-yocto_3.2": }} linux-yocto_3.2.bbappend
{{ if kernel_choice == "linux-yocto-rt_3.0": }} linux-yocto-rt_3.0.bbappend
{{ if kernel_choice == "linux-yocto-rt_3.2": }} linux-yocto-rt_3.2.bbappend
kernel-list.noinstall
$
Quoted 2 lines

PLEASE do not to this!

If everyone really hates this, I'm perfectly happy with getting rid of it - it's supposed to be a usability tool after all, and I definitely don't want to be doing the opposite...

Tom

Quoted 4 lines

Best regards,

Wolfgang Denk

#6 lore raw
In reply to Tom Zanussi

Hi Tom,

Quoted 4 lines

It may seem strange, but filenames are just text after all, and I wanted to avoid having separate 'meta-files' describing that same logic that would also have to be kept in sync, so decided it would be less error-prone to keep that information in the filenames themselves.

It indeed is a possibility to encode "more than usual" into filenames, but it certainly will have other effects also. Now that the name of the file itself contains "business logic" that is important for the system, I wonder how this interacts with other parts of the (sometimes unwritten) expectations of the development environment. One thing which immediately comes to my mind is the interaction with the version control system.

Changing "an attribute" of such a file (which is coded in the inline python code) will essentially look like the deletion and addition of a file for version control. Of course our version control (git) will be clever enough to identify the file as being the same, but still we now have an operation which was previously not used to change "business logic", i.e. renaming files, doing just that. So for tracing the history of such an item I now need to trace the "rename history" additionally to the "content history".

So in contrast to the original idea of "keeping the information in one place", we now have two places, i.e. the content and the filename and the latter is really a very uncommon place for that.

I do have a severe bad feeling in my stomach about going into this (uncharted) territory and I do not know whether the potential problems that we (yet) do not see are worth it. So I would also like you to reconsider that strategy.

Thanks
Detlev

--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: [email protected]

#7 lore raw
Quoted 24 lines · Detlev Zundel

Hi Tom,

Quoted 4 lines

It may seem strange, but filenames are just text after all, and I wanted to avoid having separate 'meta-files' describing that same logic that would also have to be kept in sync, so decided it would be less error-prone to keep that information in the filenames themselves.

It indeed is a possibility to encode "more than usual" into filenames, but it certainly will have other effects also. Now that the name of the file itself contains "business logic" that is important for the system, I wonder how this interacts with other parts of the (sometimes unwritten) expectations of the development environment. One thing which immediately comes to my mind is the interaction with the version control system.

Changing "an attribute" of such a file (which is coded in the inline python code) will essentially look like the deletion and addition of a file for version control. Of course our version control (git) will be clever enough to identify the file as being the same, but still we now have an operation which was previously not used to change "business logic", i.e. renaming files, doing just that. So for tracing the history of such an item I now need to trace the "rename history" additionally to the "content history".

Quoted 9 lines

So in contrast to the original idea of "keeping the information in one place", we now have two places, i.e. the content and the filename and the latter is really a very uncommon place for that.

I do have a severe bad feeling in my stomach about going into this (uncharted) territory and I do not know whether the potential problems that we (yet) do not see are worth it. So I would also like you to reconsider that strategy.

Yes, I think it would make sense for various reasons to get rid of it if possible - it's not central to the mechanism, just convenient.

So the current uses of it are the following:

- having a particular filename match the machine name:

    ./powerpc/conf/machine/{{=machine}}.conf

- having a particular filename conditionally included or not:

    ./i386/recipes-kernel/linux/{{ if kernel_choice ==

"linux-yocto_3.2": }} linux-yocto_3.2.bbappend

For those two types of cases, I think it should be possible to move that logic into the file itself using special filename and/or conditional filename directives.

The other usage is to have directory names match the machine name for instance:

./i386/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver_choice == "xserver_vesa": }} xorg.conf

In the case of directory names, since there's no file to put that directive into, and we can't use the directory name any longer, we'll have to again use some kind of special directory name directive and find some file to put it into, probably just another .noinstall file, which we already have a case of but that I wanted to get rid of. There may be a better way to do it, but we could keep those and do things that way in the worst case.

Tom

Quoted 12 lines

Thanks
Detlev

--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: [email protected]

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

#9 lore raw
In reply to a message not in the archive
Quoted 11 lines · Detlev Zundel

Hi Tom,

thanks for listening to my concerns. I have to admit though, that I wrote my mail because of my doubts about the usage of such unusal filenames, but I have yet to understand how you use the constructs.

Quoted 5 lines

Yes, I think it would make sense for various reasons to get rid of it if possible - it's not central to the mechanism, just convenient.

So the current uses of it are the following:

Quoted 8 lines
Quoted 3 lines

- having a particular filename match the machine name:

    ./powerpc/conf/machine/{{=machine}}.conf

Can you explain to me what this exactly does? Does that file match every machine name? If that is the case, then why do you need to match at all and not use a special file name?

I guess 'match' is the wrong word - basically what happens when processing the templates (and since the filenames themselves are part of the 'templates', those too), the {{=machine}} in the name is replaced with the name of the BSP. So in the final set of files, supposing you're creating a bsp named 'meta-foo', you end up with this file being named:

./powerpc/conf/machine/foo.conf

which is what's expected in the BSP layout.

Quoted 5 lines
Quoted 5 lines

- having a particular filename conditionally included or not:

    ./i386/recipes-kernel/linux/{{ if kernel_choice ==

"linux-yocto_3.2": }} linux-yocto_3.2.bbappend

Quoted 8 lines
Quoted 3 lines

For those two types of cases, I think it should be possible to move that logic into the file itself using special filename and/or conditional filename directives.

I'm not sure if I understand fully, but something like (shell-syntax) "include ${kernel_choice}.bbappend" would work - modulo ignoring errors if the file is not found of course.

So, similarly in this case, we have four files with similar names in this dir, but only one of them should be used, and which one depends on which kernel was selected. The 'template processing' uses the code in the filename to end up with that result. So your suggestion is partly fine in that we need some kind of 'include' to replace it, but it doesn't contain the logic we need to choose only that file. And it's just replacing it with something that looks like executable code anyway. But as I mentioned, I think this case also could me moved into the file itself.

Quoted 7 lines
Quoted 4 lines

The other usage is to have directory names match the machine name for instance:

./i386/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver_choice == "xserver_vesa": }} xorg.conf

I fail to understand this for the same reason than the first item ;)

This is actually a combination of the other two, replace the {{= with 'machine' but here it's a dirname, and conditionally select one of a set of files.

Again, in this case, for the file in this case, we can move the selection logic into the file, but for the dirname replacement we need another place to contain it, which we can use the aforementioned .noinstall files for, barring a better idea...

Tom

Quoted 7 lines

Cheers
Detlev

--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: [email protected]

#10 lore raw
In reply to Tom Zanussi

Hi Tom,

thanks for listening to my concerns. I have to admit though, that I wrote my mail because of my doubts about the usage of such unusal filenames, but I have yet to understand how you use the constructs.

Quoted 8 lines

Yes, I think it would make sense for various reasons to get rid of it if possible - it's not central to the mechanism, just convenient.

So the current uses of it are the following:

- having a particular filename match the machine name:

    ./powerpc/conf/machine/{{=machine}}.conf

Can you explain to me what this exactly does? Does that file match every machine name? If that is the case, then why do you need to match at all and not use a special file name?

Quoted 8 lines

- having a particular filename conditionally included or not:

    ./i386/recipes-kernel/linux/{{ if kernel_choice ==

"linux-yocto_3.2": }} linux-yocto_3.2.bbappend

For those two types of cases, I think it should be possible to move that logic into the file itself using special filename and/or conditional filename directives.

I'm not sure if I understand fully, but something like (shell-syntax) "include ${kernel_choice}.bbappend" would work - modulo ignoring errors if the file is not found of course.

Quoted 4 lines

The other usage is to have directory names match the machine name for instance:

./i386/recipes-graphics/xorg-xserver/xserver-xf86-config/{{=machine}}/{{ if xserver_choice == "xserver_vesa": }} xorg.conf

I fail to understand this for the same reason than the first item ;)

Cheers
Detlev

--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: [email protected]