devicetree.bbclass: Allow selection of dts files to build
PATCH v2 1/1 devicetree.bbclass: Allow selection of dts files to build
Add DT_FILES variable to allow the user of the class to select specific dts files to build. This is useful for packages featuring dts files for multiple machines.
Since many machine configs contain a list of dtb files (e.g. KERNEL_DEVICETREE), DT_FILES works with both dts and dtb files.
--- meta/classes-recipe/devicetree.bbclass | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
meta/classes-recipe/devicetree.bbclass +9 -1
@@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \ DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion." DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}" -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)." +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory." DT_FILES_PATH ?= "${S}" +DT_FILES[doc] = "Space-separated list of dts or dtb files to build. If empty, all dts files are built." +DT_FILES ?= "" DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot." DT_PADDING_SIZE ??= "0x3000"
@@ -125,7 +127,10 @@ def devicetree_compile(dtspath, includes, d): subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) python devicetree_do_compile() { + import re includes = expand_includes("DT_INCLUDE", d) + dtfiles = d.getVar("DT_FILES") + filter = [ re.sub("dtbo?", "dts", dtfile) for dtfile in dtfiles.split() ] listpath = d.getVar("DT_FILES_PATH") for dts in os.listdir(listpath): dtspath = os.path.join(listpath, dts)
@@ -134,6 +139,9 @@ python devicetree_do_compile() { continue # skip non-.dts files and non-overlay files except: continue # skip if can't determine if overlay + if filter: + if not(dts in filter): + continue # skip if dts not in defined filter devicetree_compile(dtspath, includes, d) }
Signature · 1 line
2.30.2
Quoted 18 lines · Petr Kubizňák
Add
DT_FILESvariable to allow the user of the class to select specific dts files to build. This is useful for packages featuring dts files for multiple machines.Since many machine configs contain a list of dtb files (e.g.
KERNEL_DEVICETREE),DT_FILESworks with both dts and dtb files.Signed-off-by Petr Kubizňák <[email protected]>--- meta/classes-recipe/devicetree.bbclass | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)meta/classes-recipe/devicetree.bbclass +0
-0
meta/classes-recipe/devicetree.bbclass KERNEL_INCLUDE ??= " \ · line 58
Quoted 7 lines
meta/classes-recipe/devicetree.bbclass +2
-1@@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \ DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion." DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}" -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)." +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory." DT_FILES_PATH ?= "${S}" +DT_FILES[doc] = "Space-separated list of dts or dtb files to build. If empty, all dts files are built."
Worth noting in the doc string that DT_FILES paths are relative to DT_FILES_PATH.
meta/classes-recipe/devicetree.bbclass def devicetree_compile(dtspath, includes, d): · line 132
Quoted 11 lines
meta/classes-recipe/devicetree.bbclass +3
-0+DT_FILES ?= "" DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot." DT_PADDING_SIZE ??= "0x3000"@@ -125,7 +127,10 @@ def devicetree_compile(dtspath, includes, d): subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) python devicetree_do_compile() { + import re includes = expand_includes("DT_INCLUDE", d) + dtfiles = d.getVar("DT_FILES")
If this populates the variable dtfiles with DT_FILES already split it can fall back to os.listdir if empty, this avoids the need for the "if filter..." logic within the loop:
dtfiles = d.getVar("DT_FILES").split() dtfiles = [<renaming for dtb> for dtfiles in dtfiles] for dts in dtfiles or os.listdir(listpath):
meta/classes-recipe/devicetree.bbclass def devicetree_compile(dtspath, includes, d): · line 133
Quoted 1 line
meta/classes-recipe/devicetree.bbclass +1
-0+ filter = [ re.sub("dtbo?", "dts", dtfile) for dtfile in dtfiles.split() ]
This regex is a bit problematic, it will substitute any part of the filename and not just the extension suffix. E.g. "test-dtbo.dts" -> "test-dts.dts". The expression might need to be something similar to "\.dtbo?$".
I am also curious why you need to handle this mapping of .dtb -> .dts. Do you have an example recipe that is using this class in this way? Because the ".dtb" in KERNEL_DEVICETREE is specifically because the KERNEL_DEVICETREE variable is pointing at kernel makefile targets.
Regards,
Nathan
Quoted 25 lines
meta/classes-recipe/devicetree.bbclass +3
-1listpath = d.getVar("DT_FILES_PATH") for dts in os.listdir(listpath): dtspath = os.path.join(listpath, dts)@@ -134,6 +139,9 @@ python devicetree_do_compile() { continue # skip non-.dts files and non-overlay files except: continue # skip if can't determine if overlay + if filter: + if not(dts in filter): + continue # skip if dts not in defined filter devicetree_compile(dtspath, includes, d) } --2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#180281): https://lists.openembedded.org/g/openembedded-core/message/180281
Mute This Topic: https://lists.openembedded.org/mt/98406619/3616963
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Hi Nathan,
Thank you for great suggestions. All of them are valid and I will update the patch once we discuss the .dtb -> .dts mapping.
Quoted 4 lines
I am also curious why you need to handle this mapping of .dtb -> .dts. Do you have an example recipe that is using this class in this way? Because the ".dtb" in
KERNEL_DEVICETREEis specifically because theKERNEL_DEVICETREEvariable is pointing at kernel makefile targets.
I don't have an example of a particular recipe, but how about EXTERNAL_KERNEL_DEVICETREE variable, used in kernel-fitimage.bbclass? Albeit I don't use this variable in my recipes, I believe it could be used for both classes now with simple assignment DT_FILES = EXTERNAL_KERNEL_DEVICETREE.
And if for whatever reason someone uses a custom variable (like me), they would still expect dtb's rather than dts's. I didn't find any example of using lists of dts files. Contrary, kernel-devicetree.bbclass does the same translation dtb->dts and it even warns about dts being provided (see normalize_dtb()). I didn't want to be that strict here (if someone has a list of dts files, while not to use it directly) but in general, I expect a list of dtb files will be much more common.
Kind Regards,
Petr
________________________________________
From: Nathan Rossi <[email protected]>
Sent: Friday, April 21, 2023 1:44 PM
To: Petr Kubizňák - 2N
Cc: [email protected]
Subject: Re: [OE-core][PATCH v2] devicetree.bbclass: Allow selection of dts files to build
Quoted 18 lines · Petr Kubizňák
Add
DT_FILESvariable to allow the user of the class to select specific dts files to build. This is useful for packages featuring dts files for multiple machines.Since many machine configs contain a list of dtb files (e.g.
KERNEL_DEVICETREE),DT_FILESworks with both dts and dtb files.Signed-off-by Petr Kubizňák <[email protected]>--- meta/classes-recipe/devicetree.bbclass | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)meta/classes-recipe/devicetree.bbclass +0
-0
meta/classes-recipe/devicetree.bbclass KERNEL_INCLUDE ??= " \ · line 58
Quoted 7 lines
meta/classes-recipe/devicetree.bbclass +2
-1@@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \ DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion." DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}" -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)." +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory." DT_FILES_PATH ?= "${S}" +DT_FILES[doc] = "Space-separated list of dts or dtb files to build. If empty, all dts files are built."
Worth noting in the doc string that DT_FILES paths are relative to DT_FILES_PATH.
meta/classes-recipe/devicetree.bbclass def devicetree_compile(dtspath, includes, d): · line 132
Quoted 11 lines
meta/classes-recipe/devicetree.bbclass +3
-0+DT_FILES ?= "" DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot." DT_PADDING_SIZE ??= "0x3000"@@ -125,7 +127,10 @@ def devicetree_compile(dtspath, includes, d): subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) python devicetree_do_compile() { + import re includes = expand_includes("DT_INCLUDE", d) + dtfiles = d.getVar("DT_FILES")
If this populates the variable dtfiles with DT_FILES already split it can fall back to os.listdir if empty, this avoids the need for the "if filter..." logic within the loop:
dtfiles = d.getVar("DT_FILES").split() dtfiles = [<renaming for dtb> for dtfiles in dtfiles] for dts in dtfiles or os.listdir(listpath):
meta/classes-recipe/devicetree.bbclass def devicetree_compile(dtspath, includes, d): · line 133
Quoted 1 line
meta/classes-recipe/devicetree.bbclass +1
-0+ filter = [ re.sub("dtbo?", "dts", dtfile) for dtfile in dtfiles.split() ]
This regex is a bit problematic, it will substitute any part of the filename and not just the extension suffix. E.g. "test-dtbo.dts" -> "test-dts.dts". The expression might need to be something similar to "\.dtbo?$".
I am also curious why you need to handle this mapping of .dtb -> .dts. Do you have an example recipe that is using this class in this way? Because the ".dtb" in KERNEL_DEVICETREE is specifically because the KERNEL_DEVICETREE variable is pointing at kernel makefile targets.
Regards,
Nathan
Quoted 25 lines
meta/classes-recipe/devicetree.bbclass +3
-1listpath = d.getVar("DT_FILES_PATH") for dts in os.listdir(listpath): dtspath = os.path.join(listpath, dts)@@ -134,6 +139,9 @@ python devicetree_do_compile() { continue # skip non-.dts files and non-overlay files except: continue # skip if can't determine if overlay + if filter: + if not(dts in filter): + continue # skip if dts not in defined filter devicetree_compile(dtspath, includes, d) } --2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#180281): https://lists.openembedded.org/g/openembedded-core/message/180281
Mute This Topic: https://lists.openembedded.org/mt/98406619/3616963
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 1 line
kernel-devicetree.bbclass does the same translation
dtb->dts
I mean dts->dtb, of course. The point is dtb's are expected on the input.
Petr
________________________________________
From: Petr Kubizňák - 2N <[email protected]>
Sent: Friday, April 21, 2023 2:30 PM
To: Nathan Rossi
Cc: [email protected]
Subject: Re: [OE-core][PATCH v2] devicetree.bbclass: Allow selection of dts files to build
Hi Nathan,
Thank you for great suggestions. All of them are valid and I will update the patch once we discuss the .dtb -> .dts mapping.
Quoted 4 lines
I am also curious why you need to handle this mapping of .dtb -> .dts. Do you have an example recipe that is using this class in this way? Because the ".dtb" in
KERNEL_DEVICETREEis specifically because theKERNEL_DEVICETREEvariable is pointing at kernel makefile targets.
I don't have an example of a particular recipe, but how about EXTERNAL_KERNEL_DEVICETREE variable, used in kernel-fitimage.bbclass? Albeit I don't use this variable in my recipes, I believe it could be used for both classes now with simple assignment DT_FILES = EXTERNAL_KERNEL_DEVICETREE.
And if for whatever reason someone uses a custom variable (like me), they would still expect dtb's rather than dts's. I didn't find any example of using lists of dts files. Contrary, kernel-devicetree.bbclass does the same translation dtb->dts and it even warns about dts being provided (see normalize_dtb()). I didn't want to be that strict here (if someone has a list of dts files, while not to use it directly) but in general, I expect a list of dtb files will be much more common.
Kind Regards,
Petr
________________________________________
From: Nathan Rossi <[email protected]>
Sent: Friday, April 21, 2023 1:44 PM
To: Petr Kubizňák - 2N
Cc: [email protected]
Subject: Re: [OE-core][PATCH v2] devicetree.bbclass: Allow selection of dts files to build
Quoted 18 lines · Petr Kubizňák
Add
DT_FILESvariable to allow the user of the class to select specific dts files to build. This is useful for packages featuring dts files for multiple machines.Since many machine configs contain a list of dtb files (e.g.
KERNEL_DEVICETREE),DT_FILESworks with both dts and dtb files.Signed-off-by Petr Kubizňák <[email protected]>--- meta/classes-recipe/devicetree.bbclass | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)meta/classes-recipe/devicetree.bbclass +0
-0
meta/classes-recipe/devicetree.bbclass KERNEL_INCLUDE ??= " \ · line 58
Quoted 7 lines
meta/classes-recipe/devicetree.bbclass +2
-1@@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \ DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion." DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}" -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)." +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory." DT_FILES_PATH ?= "${S}" +DT_FILES[doc] = "Space-separated list of dts or dtb files to build. If empty, all dts files are built."
Worth noting in the doc string that DT_FILES paths are relative to DT_FILES_PATH.
meta/classes-recipe/devicetree.bbclass def devicetree_compile(dtspath, includes, d): · line 132
Quoted 11 lines
meta/classes-recipe/devicetree.bbclass +3
-0+DT_FILES ?= "" DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot." DT_PADDING_SIZE ??= "0x3000"@@ -125,7 +127,10 @@ def devicetree_compile(dtspath, includes, d): subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) python devicetree_do_compile() { + import re includes = expand_includes("DT_INCLUDE", d) + dtfiles = d.getVar("DT_FILES")
If this populates the variable dtfiles with DT_FILES already split it can fall back to os.listdir if empty, this avoids the need for the "if filter..." logic within the loop:
dtfiles = d.getVar("DT_FILES").split() dtfiles = [<renaming for dtb> for dtfiles in dtfiles] for dts in dtfiles or os.listdir(listpath):
meta/classes-recipe/devicetree.bbclass def devicetree_compile(dtspath, includes, d): · line 133
Quoted 1 line
meta/classes-recipe/devicetree.bbclass +1
-0+ filter = [ re.sub("dtbo?", "dts", dtfile) for dtfile in dtfiles.split() ]
This regex is a bit problematic, it will substitute any part of the filename and not just the extension suffix. E.g. "test-dtbo.dts" -> "test-dts.dts". The expression might need to be something similar to "\.dtbo?$".
I am also curious why you need to handle this mapping of .dtb -> .dts. Do you have an example recipe that is using this class in this way? Because the ".dtb" in KERNEL_DEVICETREE is specifically because the KERNEL_DEVICETREE variable is pointing at kernel makefile targets.
Regards,
Nathan
Quoted 25 lines
meta/classes-recipe/devicetree.bbclass +3
-1listpath = d.getVar("DT_FILES_PATH") for dts in os.listdir(listpath): dtspath = os.path.join(listpath, dts)@@ -134,6 +139,9 @@ python devicetree_do_compile() { continue # skip non-.dts files and non-overlay files except: continue # skip if can't determine if overlay + if filter: + if not(dts in filter): + continue # skip if dts not in defined filter devicetree_compile(dtspath, includes, d) } --2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#180281): https://lists.openembedded.org/g/openembedded-core/message/180281
Mute This Topic: https://lists.openembedded.org/mt/98406619/3616963
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 18 lines · On Fri, 21 Apr 2023 at 22:47, Petr Kubizňák - 2N <[email protected]> wrote:
Quoted 2 lines
kernel-devicetree.bbclass does the same translation
dtb->dtsI mean
dts->dtb, of course. The point is dtb's are expected on the input.Petr
________________________________________
From: Petr Kubizňák - 2N <[email protected]>
Sent: Friday, April 21, 2023 2:30 PM
To: Nathan Rossi
Cc: [email protected]
Subject: Re: [OE-core][PATCH v2] devicetree.bbclass: Allow selection of dts files to buildHi Nathan,
Thank you for great suggestions. All of them are valid and I will update the patch once we discuss the .dtb -> .dts mapping.
Quoted 8 lines
Quoted 4 lines
I am also curious why you need to handle this mapping of .dtb -> .dts. Do you have an example recipe that is using this class in this way? Because the ".dtb" in
KERNEL_DEVICETREEis specifically because theKERNEL_DEVICETREEvariable is pointing at kernel makefile targets.I don't have an example of a particular recipe, but how about
EXTERNAL_KERNEL_DEVICETREEvariable, used in kernel-fitimage.bbclass? Albeit I don't use this variable in my recipes, I believe it could be used for both classes now with simple assignmentDT_FILES=EXTERNAL_KERNEL_DEVICETREE.And if for whatever reason someone uses a custom variable (like me), they would still expect dtb's rather than dts's. I didn't find any example of using lists of dts files. Contrary, kernel-devicetree.bbclass does the same translation
dtb->dtsand it even warns about dts being provided (seenormalize_dtb()). I didn't want to be that strict here (if someone has a list of dts files, while not to use it directly) but in general, I expect a list of dtb files will be much more common.
So just trying to make the DT_FILES somewhat consistent with the KERNEL_DEVICETREE, seems fine. I was only curious in case you had some special path handling that might be better solved by overriding the devicetree_do_compile and calling the associated compile functions directly.
The v3 patch looks good :).
Thanks,
Nathan
Quoted 101 lines
Kind Regards,
Petr________________________________________
From: Nathan Rossi <[email protected]>
Sent: Friday, April 21, 2023 1:44 PM
To: Petr Kubizňák - 2N
Cc: [email protected]
Subject: Re: [OE-core][PATCH v2] devicetree.bbclass: Allow selection of dts files to buildQuoted 18 lines · Petr Kubizňák
Add
DT_FILESvariable to allow the user of the class to select specific dts files to build. This is useful for packages featuring dts files for multiple machines.Since many machine configs contain a list of dtb files (e.g.
KERNEL_DEVICETREE),DT_FILESworks with both dts and dtb files.Signed-off-by Petr Kubizňák <[email protected]>--- meta/classes-recipe/devicetree.bbclass | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)meta/classes-recipe/devicetree.bbclass +0
-0meta/classes-recipe/devicetree.bbclass KERNEL_INCLUDE ??= " \ · line 58
Quoted 7 lines
meta/classes-recipe/devicetree.bbclass +2
-1@@ -53,8 +53,10 @@ KERNEL_INCLUDE ??= " \ DT_INCLUDE[doc] = "Search paths to be made available to both the device tree compiler and preprocessor for inclusion." DT_INCLUDE ?= "${DT_FILES_PATH} ${KERNEL_INCLUDE}" -DT_FILES_PATH[doc] = "Defaults to source directory, can be used to select dts files that are not in source (e.g. generated)." +DT_FILES_PATH[doc] = "Path to the directory containing dts files to build. Defaults to source directory." DT_FILES_PATH ?= "${S}" +DT_FILES[doc] = "Space-separated list of dts or dtb files to build. If empty, all dts files are built."Worth noting in the doc string that
DT_FILESpaths are relative toDT_FILES_PATH.meta/classes-recipe/devicetree.bbclass def devicetree_compile(dtspath, includes, d): · line 132
Quoted 11 lines
meta/classes-recipe/devicetree.bbclass +3
-0+DT_FILES ?= "" DT_PADDING_SIZE[doc] = "Size of padding on the device tree blob, used as extra space typically for additional properties during boot." DT_PADDING_SIZE ??= "0x3000"@@ -125,7 +127,10 @@ def devicetree_compile(dtspath, includes, d): subprocess.run(dtcargs, check = True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) python devicetree_do_compile() { + import re includes = expand_includes("DT_INCLUDE", d) + dtfiles = d.getVar("DT_FILES")If this populates the variable dtfiles with
DT_FILESalready split it can fall back to os.listdir if empty, this avoids the need for the "if filter..." logic within the loop:dtfiles = d.getVar("DT_FILES").split() dtfiles = [<renaming for dtb> for dtfiles in dtfiles] for dts in dtfiles or
os.listdir(listpath):meta/classes-recipe/devicetree.bbclass def devicetree_compile(dtspath, includes, d): · line 133
Quoted 1 line
meta/classes-recipe/devicetree.bbclass +1
-0+ filter = [ re.sub("dtbo?", "dts", dtfile) for dtfile in dtfiles.split() ]This regex is a bit problematic, it will substitute any part of the filename and not just the extension suffix. E.g. "
test-dtbo.dts" -> "test-dts.dts". The expression might need to be something similar to "\.dtbo?$".I am also curious why you need to handle this mapping of .dtb -> .dts. Do you have an example recipe that is using this class in this way? Because the ".dtb" in
KERNEL_DEVICETREEis specifically because theKERNEL_DEVICETREEvariable is pointing at kernel makefile targets.Regards,
NathanQuoted 25 lines
meta/classes-recipe/devicetree.bbclass +3
-1listpath = d.getVar("DT_FILES_PATH") for dts in os.listdir(listpath): dtspath = os.path.join(listpath, dts)@@ -134,6 +139,9 @@ python devicetree_do_compile() { continue # skip non-.dts files and non-overlay files except: continue # skip if can't determine if overlay + if filter: + if not(dts in filter): + continue # skip if dts not in defined filter devicetree_compile(dtspath, includes, d) } --2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#180281): https://lists.openembedded.org/g/openembedded-core/message/180281
Mute This Topic: https://lists.openembedded.org/mt/98406619/3616963
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-