Skip to contentSkip to conversation

externalsrc:git submodule--helper list unsupported

PATCH In review [email protected] 2 participants 2 messages Sep 28, 2022 – Oct 11, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 externalsrc:git submodule--helper list unsupported

From: John Edward Broadbent <[email protected]>

Git has removed support for "git submodule--helper list".
https://github.com/git/git/commit/31955475d1c283120d5d84247eb3fd55d9f5fdd9

This change provides an alternate method for gathering the submodules information.

Tested:
Build recipes with and without submodules

---
 meta/classes-recipe/externalsrc.bbclass | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
meta/classes-recipe/externalsrc.bbclass +10 -9
index ce753fce76..06a9548a20 100644
@@ -230,15 +230,16 @@ def srctree_hash_files(d, srcdir=None):
             env['GIT_INDEX_FILE'] = tmp_index.name
             subprocess.check_output(['git', 'add', '-A', '.'], cwd=s_dir, env=env)
             git_sha1 = subprocess.check_output(['git', 'write-tree'], cwd=s_dir, env=env).decode("utf-8")
-            submodule_helper = subprocess.check_output(['git', 'submodule--helper', 'list'], cwd=s_dir, env=env).decode("utf-8")
-            for line in submodule_helper.splitlines():
-                module_dir = os.path.join(s_dir, line.rsplit(maxsplit=1)[1])
-                if os.path.isdir(module_dir):
-                    proc = subprocess.Popen(['git', 'add', '-A', '.'], cwd=module_dir, env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
-                    proc.communicate()
-                    proc = subprocess.Popen(['git', 'write-tree'], cwd=module_dir, env=env, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
-                    stdout, _ = proc.communicate()
-                    git_sha1 += stdout.decode("utf-8")
+            if os.path.exists(".gitmodules"):
+                submodule_helper = subprocess.check_output(["git", "config", "--file", ".gitmodules", "--get-regexp", "path"], cwd=s_dir, env=env).decode("utf-8")
+                for line in submodule_helper.splitlines():
+                    module_dir = os.path.join(s_dir, line.rsplit(maxsplit=1)[1])
+                    if os.path.isdir(module_dir):
+                        proc = subprocess.Popen(['git', 'add', '-A', '.'], cwd=module_dir, env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+                        proc.communicate()
+                        proc = subprocess.Popen(['git', 'write-tree'], cwd=module_dir, env=env, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
+                        stdout, _ = proc.communicate()
+                        git_sha1 += stdout.decode("utf-8")
             sha1 = hashlib.sha1(git_sha1.encode("utf-8")).hexdigest()
         with open(oe_hash_file, 'w') as fobj:
             fobj.write(sha1)
Signature · 1 line
2.37.3.998.g577e59143f-goog
#1 lore raw

Hi John,

This approach doesn't seem correct, and we're observing build failures when cherry-picked onto latest dunfell.

What is the cwd for this line?  On dunfell, it's wherever bitbake was started right? Which might be totally unrelated to the recipe.
+ if os.path.exists(".gitmodules"):

Parsing git config this way
+ submodule_helper = subprocess.check_output(["git", "config", "--file", ".gitmodules", "--get-regexp", "path"], cwd=s_dir, env=env).decode("utf-8")
will only ever find submodules if srcdir/EXTERNALSRC is also the superproject top-level, unlike the previous method which found all submodules in the superproject containing the srcdir/EXTERNALSRC. externalsrc recipes may point to a subdirectory of the containing superproject (in fact, multiple externalsrc recipes may refer to different subdirectories in the same superproject).

Also, parsing git config enumerates inactive submodules, which appears to be ignored but I think should have been called out in the commit log.

Regards,
Peter Hurley