Skip to contentSkip to conversation

branch: fix remotes mapping

PATCH New Daniel Sangorrin 2 participants 5 messages Jun 20, 2019

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/3 branch: fix remotes mapping

Probably a copy&paste error that never got checked because nobody is using the mapping functionality.

---
 scripts/kernel_sec/branch.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kernel_sec/branch.py +1 -1
index 3ede64a..10f3339 100644
@@ -212,7 +212,7 @@ def get_remotes(mappings, mainline=None, stable=None):
         _get_configured_remotes(
             os.path.expanduser('~/.config/kernel-sec/remotes.yml')))
     for mapping in mappings:
-        left, right = arg.split(':', 1)
+        left, right = mapping.split(':', 1)
         remotes[left]['git_name'] = right
     if mainline:
         remotes['torvalds']['git_name'] = mainline
Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 2/3 help:remote_name: use colon instead of equal

The example and implementation use a colon (:) to specify remote mappings, but the command's help use the equal sign (=).

---
 scripts/import_stable.py   | 2 +-
 scripts/report_affected.py | 2 +-
 scripts/webview.py         | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
scripts/import_stable.py +1 -1
index b3261cf..2c88b07 100755
@@ -174,7 +174,7 @@ if __name__ == '__main__':
     parser.add_argument('--remote-name',
                         dest='remote_name', action='append', default=[],
                         help='git remote name mappings, e.g. stable:korg-stable',
-                        metavar='NAME=OTHER-NAME')
+                        metavar='NAME:OTHER-NAME')
     parser.add_argument('--mainline-remote',
                         dest='mainline_remote_name',
                         help="git remote name to use instead of 'torvalds'",
scripts/report_affected.py +1 -1
index fac0885..879c021 100755
@@ -76,7 +76,7 @@ if __name__ == '__main__':
     parser.add_argument('--remote-name',
                         dest='remote_name', action='append', default=[],
                         help='git remote name mappings, e.g. stable:korg-stable',
-                        metavar='NAME=OTHER-NAME')
+                        metavar='NAME:OTHER-NAME')
     parser.add_argument('--mainline-remote',
                         dest='mainline_remote_name',
                         help="git remote name to use instead of 'torvalds'",
scripts/webview.py +1 -1
index 9e16a0c..a3a643b 100755
@@ -206,7 +206,7 @@ if __name__ == '__main__':
     parser.add_argument('--remote-name',
                         dest='remote_name', action='append', default=[],
                         help='git remote name mappings, e.g. stable:korg-stable',
-                        metavar='NAME=OTHER-NAME')
+                        metavar='NAME:OTHER-NAME')
     parser.add_argument('--mainline-remote',
                         dest='mainline_remote_name',
                         help="git remote name to use instead of 'torvalds'",
Signature · 1 line
2.17.1
#2 lore raw

PATCH v2 3/3 remotes: check that remotes exist

In reply to Daniel Sangorrin

This patch introduces some checks for the local git repository containing the remotes configured in conf/remotes.yml

These checks occur at the beginning of import_stable.py, report_affected.py and webview.py, to make sure that the local repository is consistent with the configuration yaml files.

We also add a new script, prepare_remotes.py, that is useful to bootstrap the local git repository with the configured remote branches. Expert users may want to use their own, but the script is helpful for new users or a quickstart.

Both the checks and the new script, take repository name mappings into account.

---
 README.md                    |  5 ++++
 conf/remotes.yml             |  3 +++
 scripts/import_stable.py     |  9 +++----
 scripts/kernel_sec/branch.py | 42 ++++++++++++++++++++++++++++++++
 scripts/prepare_remotes.py   | 46 ++++++++++++++++++++++++++++++++++++
 scripts/report_affected.py   |  1 +
 scripts/webview.py           |  1 +
 7 files changed, 101 insertions(+), 6 deletions(-)
 create mode 100755 scripts/prepare_remotes.py
README.md +5 -0
index 4c5808f..576cc75 100644
@@ -23,6 +23,10 @@ this is assumed to be in `../kernel`, with remotes configured in
 stable and cip repositories. These can be overridden by command-line options
 or configuration (`~/.config/kernel-sec/remotes.yml`).
 
+* `scripts/prepare_remotes.py` - creates the local git repository
+and adds all configured remotes. You may prefer to skip this script
+and configure the repository by hand.
+
 * `scripts/import_debian.py` - import information from Debian's
 `kernel_sec` project.  It includes all issues that Debian considers
 active or that are already tracked here.
@@ -81,6 +85,7 @@ with the keys:
   branch from this remote.
 * `git_name`: (optional) The name actually used for this git
   remote, if it's different from the default.
+* `git_repo_url`: URL of the remote git repository.
 
 ## Contributions
 
conf/remotes.yml +3 -0
index 51c523d..cfaa35f 100644
@@ -1,6 +1,9 @@
 torvalds:
   commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=
+  git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
 stable:
   commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit?id=
+  git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
 cip:
   commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git/commit?id=
+  git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git
scripts/import_stable.py +3 -6
index 2c88b07..194219d 100755
@@ -31,11 +31,6 @@ BACKPORT_COMMIT_BOTTOM_RE = re.compile(
     .format(**RE_USE))
 
 
-def update(git_repo, remote_name):
-    subprocess.check_call(['git', 'remote', 'update', remote_name],
-                          cwd=git_repo)
-
-
 def get_backports(git_repo, remotes, branches, debug=False):
     backports = {}
 
@@ -140,7 +135,8 @@ def main(git_repo, remotes, debug=False):
     remote_names = set(branch['git_remote'] for branch in branches)
 
     for remote_name in remote_names:
-        update(git_repo, remotes[remote_name]['git_name'])
+        kernel_sec.branch.remote_update(
+            git_repo, remotes[remote_name]['git_name'])
     backports = get_backports(git_repo, remotes, branches, debug)
     c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)
 
@@ -190,4 +186,5 @@ if __name__ == '__main__':
     remotes = kernel_sec.branch.get_remotes(args.remote_name,
                                             mainline=args.mainline_remote_name,
                                             stable=args.stable_remote_name)
+    kernel_sec.branch.check_git_repo(args.git_repo, remotes)
     main(args.git_repo, remotes, args.debug)
scripts/kernel_sec/branch.py +42 -0

55 lines not on this page · raw message

scripts/prepare_remotes.py new file +46 -0

46 lines not on this page · raw message

scripts/report_affected.py +1 -0
index 879c021..d2f1f22 100755
@@ -100,5 +100,6 @@ if __name__ == '__main__':
     remotes = kernel_sec.branch.get_remotes(args.remote_name,
                                             mainline=args.mainline_remote_name,
                                             stable=args.stable_remote_name)
+    kernel_sec.branch.check_git_repo(args.git_repo, remotes)
     main(args.git_repo, remotes,
          args.only_fixed_upstream, args.include_ignored, *args.branches)
scripts/webview.py +1 -0
index a3a643b..e4b36f6 100755
@@ -219,6 +219,7 @@ if __name__ == '__main__':
     remotes = kernel_sec.branch.get_remotes(args.remote_name,
                                             mainline=args.mainline_remote_name,
                                             stable=args.stable_remote_name)
+    kernel_sec.branch.check_git_repo(git_repo, remotes)
 
     conf = {
         '/static/style.css': {
Signature · 1 line
2.17.1
#3 lore raw

PATCH v2 3/3 remotes: check that remotes exist

On Thu, 2019-06-20 at 14:51 +0900, Daniel Sangorrin wrote:
[...]

Quoted 2 lines
scripts/kernel_sec/branch.py +0 -0

[...]

Quoted 9 lines
scripts/kernel_sec/branch.py +9 -0
+def check_git_repo(git_repo, remotes, create=False):
+????if create:
+????????if os.path.isdir(git_repo):
+????????????msg = "directory %r already exists" % git_repo
+????????????raise argparse.ArgumentTypeError(msg)
+????????else:
+????????????os.mkdir(git_repo)
+????????????subprocess.check_call(['git', 'init', '.'], cwd=git_repo)
+

scripts/kernel_sec/branch.py

Quoted 10 lines
scripts/kernel_sec/branch.py +10 -0
+????????for key in remotes.keys():
+????????????remote = remotes[key]??# __getitem__ will add git_name
+????????????remote_add(git_repo, remote['git_name'], remote['git_repo_url'])
+????else:
+????????if not os.path.isdir(git_repo):
+????????????msg = "directory %r not present" % git_repo
+????????????raise argparse.ArgumentTypeError(msg)
+????????if not os.path.isdir(os.path.join(git_repo, '.git')):
+????????????msg = "directory %r is not a git repository" % git_repo
+????????????raise argparse.ArgumentTypeError(msg)

This isn't compatible with the use of git worktrees (which I use). In a worktree, .git is a regular file that refers back to the repository.

So I think this check should use exists() instead of isdir().

Also, why does this function raise argparse.ArgumentTypeError? The doc-string for that type says it's "An error from trying to convert a command line string to a type." It doesn't seem to be suitable for reporting problems with the thing that an argument refers to.

[...]

Quoted 4 lines
scripts/prepare_remotes.py new file +1 -0
@@ -0,0 +1,46 @@
+#!/usr/bin/python3

scripts/prepare_remotes.py line 11

Quoted 10 lines
scripts/prepare_remotes.py new file +10 -0
+
+# Copyright 2019 Toshiba corp.
+# Based on import_stable.py by Codethink Ltd.
+#
+# This script is distributed under the terms and conditions of the GNU General
+# Public License, Version 3 or later. See?http://www.gnu.org/copyleft/gpl.html
+# for details.
+
+# Helper script that prepares the local git repository with the configured
+# remote branches

[...]

Perhaps prepare_remotes.py should also run "git remote update"? Without that, the repository is still not actually useful.

Ben.

Signature · 3 lines
Ben Hutchings, Software Developer                ?        Codethink Ltd
https://www.codethink.co.uk/                 Dale House, 35 Dale Street
                                     Manchester, M1 2HF, United Kingdom
#4 lore raw

PATCH v2 1/3 branch: fix remotes mapping

In reply to Daniel Sangorrin

I've applied this and the following patch, as they're obviously correct.

Ben.

Quoted 23 lines · Daniel Sangorrin

Probably a copy&paste error that never got checked because nobody is using the mapping functionality.

---
?scripts/kernel_sec/branch.py | 2 +-
?1 file changed, 1 insertion(+), 1 deletion(-)

scripts/kernel_sec/branch.py +0 -0
index 3ede64a..10f3339 100644
@@ -212,7 +212,7 @@ def get_remotes(mappings, mainline=None,

stable=None):
?????????_get_configured_remotes( ?????????????os.path.expanduser('~/.config/kernel-sec/remotes.yml'))) ?????for mapping in mappings:
-????????left, right = arg.split(':', 1)
+????????left, right = mapping.split(':', 1) ?????????remotes[left]['git_name'] = right
?????if mainline:
?????????remotes['torvalds']['git_name'] = mainline

Signature · 3 lines
Ben Hutchings, Software Developer                ?        Codethink Ltd
https://www.codethink.co.uk/                 Dale House, 35 Dale Street
                                     Manchester, M1 2HF, United Kingdom