Split split_and_strip_files()
cover Split split_and_strip_files()
This is my third attempt to make it possible for me to hook into the split_and_strip_files() function. We need this to be able to rename the debug files for some packages to avoid conflicts when installing them.
In this version, the first patch splits out the part of split_and_strip_files() that is responsible for setting up the variables used to implement the various debug splitting methods specified by PACKAGE_DEBUG_SPLIT_STYLE into a new function, package_debug_vars(). This I believe to be a good thing in itself as that function is way too long anyway.
The second and third patch are not needed to allow me to hook into the split_and_strip_files() function, but they simplify the APIs used for passing the variables returned by package_debug_vars() around.
With these patches in place, I can now wrap the package_debug_vars() function with the following local wrapper package.bbclass:
require ${COREBASE}/meta/classes/package.bbclass
PACKAGE_DEBUG_FILE_SUFFIX ??= ""
# Provide the original package_debug_vars() as org_package_debug_vars().org_package_debug_vars := "${@d.getVar('package_debug_vars', False).replace('package_debug_vars', 'org_package_debug_vars')}"
python () {
bb.methodpool.insert_method("", d.getVar("org_package_debug_vars", False), "<code>", 0)
}# This version of package_debug_vars() wraps the original package_debug_vars()
# by calling org_package_debug_vars() and then appends PACKAGE_DEBUG_FILE_SUFFIX
# to dv["append"] and dv["staticappend"].
def package_debug_vars(d):
dv = org_package_debug_vars(d) dv["append"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX')
dv["staticappend"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX')
return dvThe definition of the org_package_debug_vars() function is not very pretty, but it seems to work, and this solution means a heck of a lot less maintenance than to try to maintain a backported version of the over 250 lines long split_and_strip_files() function (as it was before my changes).
//Peter
The following changes since commit 74a8a74a553a33dc5f41939f8070d75e6d57d3da:
busybox: Add shell arithmetic to work with poky-tiny (2022-02-01 07:32:08 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/poky-contrib pkj/package_debug_vars
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/package_debug_vars
Peter Kjellerstedt (3):
package.bbclass: Split out package_debug_vars from
split_and_strip_files
package.bbclass: Make package_debug_vars() return a dict package.bbclass: Pass dv (debug_vars) around instead of individual
vars
meta/classes/package.bbclass | 127 +++++++++++++++++++---------------- 1 file changed, 70 insertions(+), 57 deletions(-)
PATCH 1/3 package.bbclass: Split out package_debug_vars from split_and_strip_files
The split_and_strip_files() function is big and hard to follow. This takes a small step to improve that by splitting out the part that sets up the variables used to implement the various debug splitting methods specified by PACKAGE_DEBUG_SPLIT_STYLE.
--- meta/classes/package.bbclass | 29 ++++++++++++++++++----------- 1 file changed, 18 insertions(+), 11 deletions(-)
meta/classes/package.bbclass +18 -11
@@ -1065,17 +1065,7 @@ python fixup_perms () { fix_perms(each_file, fs_perms_table[dir].fmode, fs_perms_table[dir].fuid, fs_perms_table[dir].fgid, dir) } -python split_and_strip_files () { - import stat, errno - import subprocess - - dvar = d.getVar('PKGD') - pn = d.getVar('PN') - hostos = d.getVar('HOST_OS') - - oldcwd = os.getcwd() - os.chdir(dvar) - +def package_debug_vars(d): # We default to '.debug' style if d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-file-directory': # Single debug-file-directory style debug info
@@ -1113,6 +1103,23 @@ python split_and_strip_files () { debugstaticlibdir = "" debugsrcdir = "/usr/src/debug" + return (debugappend, debugstaticappend, debugdir, debugstaticdir, + debuglibdir, debugstaticlibdir, debugsrcdir) + +python split_and_strip_files () { + import stat, errno + import subprocess + + dvar = d.getVar('PKGD') + pn = d.getVar('PN') + hostos = d.getVar('HOST_OS') + + oldcwd = os.getcwd() + os.chdir(dvar) + + debugappend, debugstaticappend, debugdir, debugstaticdir, \ + debuglibdir, debugstaticlibdir, debugsrcdir = package_debug_vars(d) + # # First lets figure out all of the files we may have to process ... do this only once! #
PATCH 2/3 package.bbclass: Make package_debug_vars() return a dict
In reply to Peter KjellerstedtIt simplifies the API to return one dict instead of seven strings.
--- meta/classes/package.bbclass | 96 +++++++++++++++++++----------------- 1 file changed, 51 insertions(+), 45 deletions(-)
meta/classes/package.bbclass +51 -45
@@ -1069,42 +1069,49 @@ def package_debug_vars(d): # We default to '.debug' style if d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-file-directory': # Single debug-file-directory style debug info - debugappend = ".debug" - debugstaticappend = "" - debugdir = "" - debugstaticdir = "" - debuglibdir = "/usr/lib/debug" - debugstaticlibdir = "/usr/lib/debug-static" - debugsrcdir = "/usr/src/debug" + debug_vars = { + "append": ".debug", + "staticappend": "", + "dir": "", + "staticdir": "", + "libdir": "/usr/lib/debug", + "staticlibdir": "/usr/lib/debug-static", + "srcdir": "/usr/src/debug", + } elif d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-without-src': # Original OE-core, a.k.a. ".debug", style debug info, but without sources in /usr/src/debug - debugappend = "" - debugstaticappend = "" - debugdir = "/.debug" - debugstaticdir = "/.debug-static" - debuglibdir = "" - debugstaticlibdir = "" - debugsrcdir = "" + debug_vars = { + "append": "", + "staticappend": "", + "dir": "/.debug", + "staticdir": "/.debug-static", + "libdir": "", + "staticlibdir": "", + "srcdir": "", + } elif d.getVar('PACKAGE_DEBUG_SPLIT_STYLE') == 'debug-with-srcpkg': - debugappend = "" - debugstaticappend = "" - debugdir = "/.debug" - debugstaticdir = "/.debug-static" - debuglibdir = "" - debugstaticlibdir = "" - debugsrcdir = "/usr/src/debug" + debug_vars = { + "append": "", + "staticappend": "", + "dir": "/.debug", + "staticdir": "/.debug-static", + "libdir": "", + "staticlibdir": "", + "srcdir": "/usr/src/debug", + } else: # Original OE-core, a.k.a. ".debug", style debug info - debugappend = "" - debugstaticappend = "" - debugdir = "/.debug" - debugstaticdir = "/.debug-static" - debuglibdir = "" - debugstaticlibdir = "" - debugsrcdir = "/usr/src/debug" + debug_vars = { + "append": "", + "staticappend": "", + "dir": "/.debug", + "staticdir": "/.debug-static", + "libdir": "", + "staticlibdir": "", + "srcdir": "/usr/src/debug", + } - return (debugappend, debugstaticappend, debugdir, debugstaticdir, - debuglibdir, debugstaticlibdir, debugsrcdir) + return debug_vars python split_and_strip_files () { import stat, errno
@@ -1117,8 +1124,7 @@ python split_and_strip_files () { oldcwd = os.getcwd() os.chdir(dvar) - debugappend, debugstaticappend, debugdir, debugstaticdir, \ - debuglibdir, debugstaticlibdir, debugsrcdir = package_debug_vars(d) + dv = package_debug_vars(d) # # First lets figure out all of the files we may have to process ... do this only once!
@@ -1139,9 +1145,9 @@ python split_and_strip_files () { file = os.path.join(root, f) # Skip debug files - if debugappend and file.endswith(debugappend): + if dv["append"] and file.endswith(dv["append"]): continue - if debugdir and debugdir in os.path.dirname(file[len(dvar):]): + if dv["dir"] and dv["dir"] in os.path.dirname(file[len(dvar):]): continue if file in skipfiles:
@@ -1238,11 +1244,11 @@ python split_and_strip_files () { # First lets process debug splitting # if (d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT') != '1'): - results = oe.utils.multiprocess_launch(splitdebuginfo, list(elffiles), d, extraargs=(dvar, debugdir, debuglibdir, debugappend, debugsrcdir, d)) + results = oe.utils.multiprocess_launch(splitdebuginfo, list(elffiles), d, extraargs=(dvar, dv["dir"], dv["libdir"], dv["append"], dv["srcdir"], d)) - if debugsrcdir and not hostos.startswith("mingw"): + if dv["srcdir"] and not hostos.startswith("mingw"): if (d.getVar('PACKAGE_DEBUG_STATIC_SPLIT') == '1'): - results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, debugstaticdir, debugstaticlibdir, debugstaticappend, debugsrcdir, d)) + results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, dv["staticdir"], dv["staticlibdir"], dv["staticappend"], dv["srcdir"], d)) else: for file in staticlibs: results.append( (file,source_info(file, d)) )
@@ -1261,9 +1267,9 @@ python split_and_strip_files () { target = inodes[ref][0][len(dvar):] for file in inodes[ref][1:]: src = file[len(dvar):] - dest = debuglibdir + os.path.dirname(src) + debugdir + "/" + os.path.basename(target) + debugappend + dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] fpath = dvar + dest - ftarget = dvar + debuglibdir + os.path.dirname(target) + debugdir + "/" + os.path.basename(target) + debugappend + ftarget = dvar + dv["libdir"] + os.path.dirname(target) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] bb.utils.mkdirhier(os.path.dirname(fpath)) # Only one hardlink of separated debug info file in each directory if not os.access(fpath, os.R_OK):
@@ -1273,7 +1279,7 @@ python split_and_strip_files () { # Create symlinks for all cases we were able to split symbols for file in symlinks: src = file[len(dvar):] - dest = debuglibdir + os.path.dirname(src) + debugdir + "/" + os.path.basename(src) + debugappend + dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(src) + dv["append"] fpath = dvar + dest # Skip it if the target doesn't exist try:
@@ -1289,17 +1295,17 @@ python split_and_strip_files () { lbase = os.path.basename(ltarget) ftarget = "" if lpath and lpath != ".": - ftarget += lpath + debugdir + "/" - ftarget += lbase + debugappend + ftarget += lpath + dv["dir"] + "/" + ftarget += lbase + dv["append"] if lpath.startswith(".."): ftarget = os.path.join("..", ftarget) bb.utils.mkdirhier(os.path.dirname(fpath)) #bb.note("Symlink %s -> %s" % (fpath, ftarget)) os.symlink(ftarget, fpath) - # Process the debugsrcdir if requested... + # Process the dv["srcdir"] if requested... # This copies and places the referenced sources for later debugging... - copydebugsources(debugsrcdir, sources, d) + copydebugsources(dv["srcdir"], sources, d) # # End of debug splitting #
@@ -1323,7 +1329,7 @@ python split_and_strip_files () { # Build "minidebuginfo" and reinject it back into the stripped binaries if d.getVar('PACKAGE_MINIDEBUGINFO') == '1': oe.utils.multiprocess_launch(inject_minidebuginfo, list(elffiles), d, - extraargs=(dvar, debugdir, debuglibdir, debugappend, debugsrcdir, d)) + extraargs=(dvar, dv["dir"], dv["libdir"], dv["append"], dv["srcdir"], d)) # # End of strip
PATCH 3/3 package.bbclass: Pass dv (debug_vars) around instead of individual vars
In reply to Peter KjellerstedtThis simplifies the APIs for splitdebuginfo(), splitstaticdebuginfo() and inject_minidebuginfo()
--- meta/classes/package.bbclass | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-)
meta/classes/package.bbclass +11 -11
@@ -367,7 +367,7 @@ def source_info(file, d, fatal=True): return list(debugsources) -def splitdebuginfo(file, dvar, debugdir, debuglibdir, debugappend, debugsrcdir, d): +def splitdebuginfo(file, dvar, dv, d): # Function to split a single file into two components, one is the stripped # target system binary, the other contains any debugging information. The # two files are linked to reference each other.
@@ -378,7 +378,7 @@ def splitdebuginfo(file, dvar, debugdir, debuglibdir, debugappend, debugsrcdir, import subprocess src = file[len(dvar):] - dest = debuglibdir + os.path.dirname(src) + debugdir + "/" + os.path.basename(src) + debugappend + dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(src) + dv["append"] debugfile = dvar + dest sources = []
@@ -397,7 +397,7 @@ def splitdebuginfo(file, dvar, debugdir, debuglibdir, debugappend, debugsrcdir, os.chmod(file, newmode) # We need to extract the debug src information here... - if debugsrcdir: + if dv["srcdir"]: sources = source_info(file, d) bb.utils.mkdirhier(os.path.dirname(debugfile))
@@ -412,7 +412,7 @@ def splitdebuginfo(file, dvar, debugdir, debuglibdir, debugappend, debugsrcdir, return (file, sources) -def splitstaticdebuginfo(file, dvar, debugstaticdir, debugstaticlibdir, debugstaticappend, debugsrcdir, d): +def splitstaticdebuginfo(file, dvar, dv, d): # Unlike the function above, there is no way to split a static library # two components. So to get similar results we will copy the unmodified # static library (containing the debug symbols) into a new directory.
@@ -425,7 +425,7 @@ def splitstaticdebuginfo(file, dvar, debugstaticdir, debugstaticlibdir, debugsta import shutil src = file[len(dvar):] - dest = debugstaticlibdir + os.path.dirname(src) + debugstaticdir + "/" + os.path.basename(src) + debugstaticappend + dest = dv["staticlibdir"] + os.path.dirname(src) + dv["staticdir"] + "/" + os.path.basename(src) + dv["staticappend"] debugfile = dvar + dest sources = []
@@ -442,7 +442,7 @@ def splitstaticdebuginfo(file, dvar, debugstaticdir, debugstaticlibdir, debugsta os.chmod(file, newmode) # We need to extract the debug src information here... - if debugsrcdir: + if dv["srcdir"]: sources = source_info(file, d) bb.utils.mkdirhier(os.path.dirname(debugfile))
@@ -455,7 +455,7 @@ def splitstaticdebuginfo(file, dvar, debugstaticdir, debugstaticlibdir, debugsta return (file, sources) -def inject_minidebuginfo(file, dvar, debugdir, debuglibdir, debugappend, debugsrcdir, d): +def inject_minidebuginfo(file, dvar, dv, d): # Extract just the symbols from debuginfo into minidebuginfo, # compress it with xz and inject it back into the binary in a .gnu_debugdata section. # https://sourceware.org/gdb/onlinedocs/gdb/MiniDebugInfo.html
@@ -469,7 +469,7 @@ def inject_minidebuginfo(file, dvar, debugdir, debuglibdir, debugappend, debugsr minidebuginfodir = d.expand('${WORKDIR}/minidebuginfo') src = file[len(dvar):] - dest = debuglibdir + os.path.dirname(src) + debugdir + "/" + os.path.basename(src) + debugappend + dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(src) + dv["append"] debugfile = dvar + dest minidebugfile = minidebuginfodir + src + '.minidebug' bb.utils.mkdirhier(os.path.dirname(minidebugfile))
@@ -1244,11 +1244,11 @@ python split_and_strip_files () { # First lets process debug splitting # if (d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT') != '1'): - results = oe.utils.multiprocess_launch(splitdebuginfo, list(elffiles), d, extraargs=(dvar, dv["dir"], dv["libdir"], dv["append"], dv["srcdir"], d)) + results = oe.utils.multiprocess_launch(splitdebuginfo, list(elffiles), d, extraargs=(dvar, dv, d)) if dv["srcdir"] and not hostos.startswith("mingw"): if (d.getVar('PACKAGE_DEBUG_STATIC_SPLIT') == '1'): - results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, dv["staticdir"], dv["staticlibdir"], dv["staticappend"], dv["srcdir"], d)) + results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, dv, d)) else: for file in staticlibs: results.append( (file,source_info(file, d)) )
@@ -1329,7 +1329,7 @@ python split_and_strip_files () { # Build "minidebuginfo" and reinject it back into the stripped binaries if d.getVar('PACKAGE_MINIDEBUGINFO') == '1': oe.utils.multiprocess_launch(inject_minidebuginfo, list(elffiles), d, - extraargs=(dvar, dv["dir"], dv["libdir"], dv["append"], dv["srcdir"], d)) + extraargs=(dvar, dv, d)) # # End of strip
Sorry Peter, but this still strikes me as rather heavy handed for what is a niche use case. Is it possible to simply do what needs to be done from a ROOTFS_POSTPROCESS hook?
Alex
Quoted 86 lines · Peter Kjellerstedt
This is my third attempt to make it possible for me to hook into the
split_and_strip_files()function. We need this to be able to rename the debug files for some packages to avoid conflicts when installing them.In this version, the first patch splits out the part of
split_and_strip_files()that is responsible for setting up the variables used to implement the various debug splitting methods specified byPACKAGE_DEBUG_SPLIT_STYLEinto a new function,package_debug_vars(). This I believe to be a good thing in itself as that function is way too long anyway.The second and third patch are not needed to allow me to hook into the
split_and_strip_files()function, but they simplify the APIs used for passing the variables returned bypackage_debug_vars()around.With these patches in place, I can now wrap the
package_debug_vars()function with the following local wrapper package.bbclass:require ${COREBASE}/meta/classes/package.bbclass
PACKAGE_DEBUG_FILE_SUFFIX??= ""# Provide the original
package_debug_vars()asorg_package_debug_vars().org_package_debug_vars:= "${@d.getVar('package_debug_vars', False).replace('package_debug_vars', 'org_package_debug_vars')}" python () {
bb.methodpool.insert_method("", d.getVar("org_package_debug_vars",
False), "<code>", 0)
}# This version of
package_debug_vars()wraps the originalpackage_debug_vars()# by calling org_package_debug_vars() and then appends PACKAGE_DEBUG_FILE_SUFFIX # to dv["append"] and dv["staticappend"]. def package_debug_vars(d): dv = org_package_debug_vars(d)dv["append"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX') dv["staticappend"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX') return dvThe definition of the
org_package_debug_vars()function is not very pretty, but it seems to work, and this solution means a heck of a lot less maintenance than to try to maintain a backported version of the over 250 lines longsplit_and_strip_files()function (as it was before my changes).//Peter
The following changes since commit
74a8a74a553a33dc5f41939f8070d75e6d57d3da:busybox: Add shell arithmetic to work with poky-tiny (2022-02-01
07:32:08 +0000)are available in the Git repository at:
git://git.yoctoproject.org/poky-contribpkj/package_debug_varshttp://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/package_debug_vars
Peter Kjellerstedt (3):
package.bbclass: Split outpackage_debug_varsfrom
split_and_strip_files
package.bbclass: Makepackage_debug_vars()return a dict package.bbclass: Pass dv (debug_vars) around instead of individual
varsmeta/classes/package.bbclass | 127 +++++++++++++++++++---------------- 1 file changed, 70 insertions(+), 57 deletions(-)-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#161174):
https://lists.openembedded.org/g/openembedded-core/message/161174
Mute This Topic: https://lists.openembedded.org/mt/88849297/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Why do you consider splitting a huge function heavy? Honestly, I would like to split it even further to make it more manageable.
//Peter
From: Alexander Kanavin <[email protected]>
Sent: den 2 februari 2022 09:54
To: Peter Kjellerstedt <[email protected]>
Cc: OE-core <[email protected]>
Subject: Re: [OE-core] [PATCH 0/3] Split split_and_strip_files()
Sorry Peter, but this still strikes me as rather heavy handed for what is a niche use case. Is it possible to simply do what needs to be done from a ROOTFS_POSTPROCESS hook?
Alex
On Wed, 2 Feb 2022 at 03:00, Peter Kjellerstedt <[email protected]<mailto:[email protected]>> wrote:
This is my third attempt to make it possible for me to hook into thesplit_and_strip_files() function. We need this to be able to rename
the debug files for some packages to avoid conflicts when installing
them.
In this version, the first patch splits out the part of split_and_strip_files() that is responsible for setting up the variables used to implement the various debug splitting methods specified by PACKAGE_DEBUG_SPLIT_STYLE into a new function, package_debug_vars(). This I believe to be a good thing in itself as that function is way too long anyway.
The second and third patch are not needed to allow me to hook into the split_and_strip_files() function, but they simplify the APIs used for passing the variables returned by package_debug_vars() around.
With these patches in place, I can now wrap the package_debug_vars() function with the following local wrapper package.bbclass:
require ${COREBASE}/meta/classes/package.bbclass
PACKAGE_DEBUG_FILE_SUFFIX ??= ""
# Provide the original package_debug_vars() as org_package_debug_vars().org_package_debug_vars := "${@d.getVar('package_debug_vars', False).replace('package_debug_vars', 'org_package_debug_vars')}<mailto:$%[email protected]('package_debug_vars',%20False).replace('package_debug_vars',%20'org_package_debug_vars')%7d>"
python () {
bb.methodpool.insert_method("", d.getVar("org_package_debug_vars", False), "<code>", 0)
}# This version of package_debug_vars() wraps the original package_debug_vars()
# by calling org_package_debug_vars() and then appends PACKAGE_DEBUG_FILE_SUFFIX
# to dv["append"] and dv["staticappend"].
def package_debug_vars(d):
dv = org_package_debug_vars(d) dv["append"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX')
dv["staticappend"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX')
return dvThe definition of the org_package_debug_vars() function is not very pretty, but it seems to work, and this solution means a heck of a lot less maintenance than to try to maintain a backported version of the over 250 lines long split_and_strip_files() function (as it was before my changes).
//Peter
The following changes since commit 74a8a74a553a33dc5f41939f8070d75e6d57d3da:
busybox: Add shell arithmetic to work with poky-tiny (2022-02-01 07:32:08 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/poky-contrib<http://git.yoctoproject.org/poky-contrib> pkj/package_debug_vars
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/package_debug_vars
Peter Kjellerstedt (3):
package.bbclass: Split out package_debug_vars from
split_and_strip_files
package.bbclass: Make package_debug_vars() return a dict package.bbclass: Pass dv (debug_vars) around instead of individual
vars
meta/classes/package.bbclass | 127 +++++++++++++++++++---------------- 1 file changed, 70 insertions(+), 57 deletions(-)
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161174): https://lists.openembedded.org/g/openembedded-core/message/161174
Mute This Topic: https://lists.openembedded.org/mt/88849297/1686489
Group Owner: [email protected]<mailto:openembedded-core%[email protected]>
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]<mailto:[email protected]>]
-=-=-=-=-=-=-=-=-=-=-=-
Right, the patchset is good for maintainability, my comment was more about whether it's easier to just package files in a way that doesn't clash in do_rootfs, then rearrange them in rootfs postprocessing.
Alex
Quoted 116 lines · Peter Kjellerstedt
Why do you consider splitting a huge function heavy? Honestly, I would like to split it even further to make it more manageable.
//Peter
*From:* Alexander Kanavin <[email protected]> *Sent:* den 2 februari 2022 09:54
*To:* Peter Kjellerstedt <[email protected]> *Cc:* OE-core <[email protected]> *Subject:* Re: [OE-core] [PATCH 0/3] Splitsplit_and_strip_files()Sorry Peter, but this still strikes me as rather heavy handed for what is a niche use case. Is it possible to simply do what needs to be done from a
ROOTFS_POSTPROCESShook?Alex
On Wed, 2 Feb 2022 at 03:00, Peter Kjellerstedt <
[email protected]> wrote:This is my third attempt to make it possible for me to hook into the
split_and_strip_files()function. We need this to be able to rename the debug files for some packages to avoid conflicts when installing them.In this version, the first patch splits out the part of
split_and_strip_files()that is responsible for setting up the variables used to implement the various debug splitting methods specified byPACKAGE_DEBUG_SPLIT_STYLEinto a new function,package_debug_vars(). This I believe to be a good thing in itself as that function is way too long anyway.The second and third patch are not needed to allow me to hook into the
split_and_strip_files()function, but they simplify the APIs used for passing the variables returned bypackage_debug_vars()around.With these patches in place, I can now wrap the
package_debug_vars()function with the following local wrapper package.bbclass:require ${COREBASE}/meta/classes/package.bbclass
PACKAGE_DEBUG_FILE_SUFFIX??= ""# Provide the original
package_debug_vars()asorg_package_debug_vars().org_package_debug_vars:= "${@d.getVar('package_debug_vars', False).replace('package_debug_vars', 'org_package_debug_vars')}" python () {
bb.methodpool.insert_method("", d.getVar("org_package_debug_vars",
False), "<code>", 0)
}# This version of
package_debug_vars()wraps the originalpackage_debug_vars()# by calling org_package_debug_vars() and then appends PACKAGE_DEBUG_FILE_SUFFIX # to dv["append"] and dv["staticappend"]. def package_debug_vars(d): dv = org_package_debug_vars(d)dv["append"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX') dv["staticappend"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX') return dvThe definition of the
org_package_debug_vars()function is not very pretty, but it seems to work, and this solution means a heck of a lot less maintenance than to try to maintain a backported version of the over 250 lines longsplit_and_strip_files()function (as it was before my changes).//Peter
The following changes since commit
74a8a74a553a33dc5f41939f8070d75e6d57d3da:busybox: Add shell arithmetic to work with poky-tiny (2022-02-01
07:32:08 +0000)are available in the Git repository at:
git://git.yoctoproject.org/poky-contribpkj/package_debug_varshttp://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/package_debug_vars
Peter Kjellerstedt (3):
package.bbclass: Split outpackage_debug_varsfrom
split_and_strip_files
package.bbclass: Makepackage_debug_vars()return a dict package.bbclass: Pass dv (debug_vars) around instead of individual
varsmeta/classes/package.bbclass | 127 +++++++++++++++++++---------------- 1 file changed, 70 insertions(+), 57 deletions(-)-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#161174):
https://lists.openembedded.org/g/openembedded-core/message/161174
Mute This Topic: https://lists.openembedded.org/mt/88849297/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Well, I believe it is better to package the files correctly from the start rather than having to redo the packaging later. The first variant I suggested was two added lines, compared to the ~30 lines we had in a recipe that did it after the fact, which somewhat indicates what it means to do it as part of the packaging process, compared to what it takes to do it later.
I still believe it would be better if the -dbg packages are always created in a way that two -dbg packages cannot clash with each other, since we do not split them the same way we do for target packages. That way everyone would benefit from it and it should lead to less problems. However, with the proposed changes I can solve my immediate problem, and we can continue the discussion if we can improve how -dbg packages are created for everyone separately.
//Peter
From: Alexander Kanavin <[email protected]>
Sent: den 2 februari 2022 12:33
To: Peter Kjellerstedt <[email protected]>
Cc: OE-core <[email protected]>
Subject: Re: [OE-core] [PATCH 0/3] Split split_and_strip_files()
Right, the patchset is good for maintainability, my comment was more about whether it's easier to just package files in a way that doesn't clash in do_rootfs, then rearrange them in rootfs postprocessing.
Alex
On Wed, 2 Feb 2022 at 12:20, Peter Kjellerstedt <[email protected]<mailto:[email protected]>> wrote:
Why do you consider splitting a huge function heavy? Honestly, I would like to split it even further to make it more manageable.
//Peter
From: Alexander Kanavin <[email protected]<mailto:[email protected]>>
Sent: den 2 februari 2022 09:54
To: Peter Kjellerstedt <[email protected]<mailto:[email protected]>>
Cc: OE-core <[email protected]<mailto:[email protected]>>
Subject: Re: [OE-core] [PATCH 0/3] Split split_and_strip_files()
Sorry Peter, but this still strikes me as rather heavy handed for what is a niche use case. Is it possible to simply do what needs to be done from a ROOTFS_POSTPROCESS hook?
Alex
On Wed, 2 Feb 2022 at 03:00, Peter Kjellerstedt <[email protected]<mailto:[email protected]>> wrote:
This is my third attempt to make it possible for me to hook into thesplit_and_strip_files() function. We need this to be able to rename
the debug files for some packages to avoid conflicts when installing
them.
In this version, the first patch splits out the part of split_and_strip_files() that is responsible for setting up the variables used to implement the various debug splitting methods specified by PACKAGE_DEBUG_SPLIT_STYLE into a new function, package_debug_vars(). This I believe to be a good thing in itself as that function is way too long anyway.
The second and third patch are not needed to allow me to hook into the split_and_strip_files() function, but they simplify the APIs used for passing the variables returned by package_debug_vars() around.
With these patches in place, I can now wrap the package_debug_vars() function with the following local wrapper package.bbclass:
require ${COREBASE}/meta/classes/package.bbclass
PACKAGE_DEBUG_FILE_SUFFIX ??= ""
# Provide the original package_debug_vars() as org_package_debug_vars().org_package_debug_vars := "${@d.getVar('package_debug_vars', False).replace('package_debug_vars', 'org_package_debug_vars')}<mailto:$%[email protected]('package_debug_vars',%20False).replace('package_debug_vars',%20'org_package_debug_vars')%7d>"
python () {
bb.methodpool.insert_method("", d.getVar("org_package_debug_vars", False), "<code>", 0)
}# This version of package_debug_vars() wraps the original package_debug_vars()
# by calling org_package_debug_vars() and then appends PACKAGE_DEBUG_FILE_SUFFIX
# to dv["append"] and dv["staticappend"].
def package_debug_vars(d):
dv = org_package_debug_vars(d) dv["append"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX')
dv["staticappend"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX')
return dvThe definition of the org_package_debug_vars() function is not very pretty, but it seems to work, and this solution means a heck of a lot less maintenance than to try to maintain a backported version of the over 250 lines long split_and_strip_files() function (as it was before my changes).
//Peter
The following changes since commit 74a8a74a553a33dc5f41939f8070d75e6d57d3da:
busybox: Add shell arithmetic to work with poky-tiny (2022-02-01 07:32:08 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/poky-contrib<http://git.yoctoproject.org/poky-contrib> pkj/package_debug_vars
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/package_debug_vars
Peter Kjellerstedt (3):
package.bbclass: Split out package_debug_vars from
split_and_strip_files
package.bbclass: Make package_debug_vars() return a dict package.bbclass: Pass dv (debug_vars) around instead of individual
vars
meta/classes/package.bbclass | 127 +++++++++++++++++++---------------- 1 file changed, 70 insertions(+), 57 deletions(-)
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#161174): https://lists.openembedded.org/g/openembedded-core/message/161174
Mute This Topic: https://lists.openembedded.org/mt/88849297/1686489
Group Owner: [email protected]<mailto:openembedded-core%[email protected]>
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]<mailto:[email protected]>]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 64 lines · Peter Kjellerstedt
This is my third attempt to make it possible for me to hook into the
split_and_strip_files()function. We need this to be able to rename the debug files for some packages to avoid conflicts when installing them.In this version, the first patch splits out the part of
split_and_strip_files()that is responsible for setting up the variables used to implement the various debug splitting methods specified byPACKAGE_DEBUG_SPLIT_STYLEinto a new function,package_debug_vars(). This I believe to be a good thing in itself as that function is way too long anyway.The second and third patch are not needed to allow me to hook into the
split_and_strip_files()function, but they simplify the APIs used for passing the variables returned bypackage_debug_vars()around.With these patches in place, I can now wrap the
package_debug_vars()function with the following local wrapper package.bbclass:require ${COREBASE}/meta/classes/package.bbclass
PACKAGE_DEBUG_FILE_SUFFIX??= ""# Provide the original
package_debug_vars()asorg_package_debug_vars().org_package_debug_vars:= "${@d.getVar('package_debug_vars', False).replace('package_debug_vars', 'org_package_debug_vars')}"python () { bb.methodpool.insert_method("", d.getVar("org_package_debug_vars", False), "<code>", 0) }# This version of
package_debug_vars()wraps the originalpackage_debug_vars()# by calling org_package_debug_vars() and then appends PACKAGE_DEBUG_FILE_SUFFIX # to dv["append"] and dv["staticappend"]. def package_debug_vars(d): dv = org_package_debug_vars(d)dv["append"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX') dv["staticappend"] += d.getVar('PACKAGE_DEBUG_FILE_SUFFIX') return dvThe definition of the
org_package_debug_vars()function is not very pretty, but it seems to work, and this solution means a heck of a lot less maintenance than to try to maintain a backported version of the over 250 lines longsplit_and_strip_files()function (as it was before my changes).//Peter
The following changes since commit
74a8a74a553a33dc5f41939f8070d75e6d57d3da:busybox: Add shell arithmetic to work with poky-tiny (2022-02-01 07:32:08 +0000)
are available in the Git repository at:
git://git.yoctoproject.org/poky-contribpkj/package_debug_vars
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=pkj/package_debug_varsPeter Kjellerstedt (3):
package.bbclass: Split outpackage_debug_varsfrom
split_and_strip_files
package.bbclass: Makepackage_debug_vars()return a dict package.bbclass: Pass dv (debug_vars) around instead of individual
vars
Quoted 2 lines
meta/classes/package.bbclass | 127 +++++++++++++++++++---------------- 1 file changed, 70 insertions(+), 57 deletions(-)
I know I'm often negative about patches so I just wanted to mention I like this series for a change! :)
Definitely better than the other patch with all those variables.
Cheers,
Richard