remotes: automatically add remotes from configuration file
PATCH 1/2 remotes: automatically add remotes from configuration file
Currently the user is required to create its own remotes by hand. This should not be necessary, because the information is already collected in conf/remotes.yml. For that reason, if we detect that any remote has not been added to the local repo then we will add those.
--- README.md | 5 +++-- conf/remotes.yml | 9 ++++++--- scripts/import_stable.py | 9 +++++++++ scripts/templates/issue.html | 7 ++++--- 4 files changed, 22 insertions(+), 8 deletions(-)
README.md +3 -2
@@ -77,8 +77,9 @@ These files, if they exist, contain a mapping where the keys are default git remote names. The values are also mappings, with the keys: -* `commit_url_prefix`: URL prefix for browsing a commit on a - branch from this remote. +* `git_repo_url`: URL of the remote repository. +* `commit_url_suffix`: URL suffix that gets appended to `git_repo_url` + for browsing a commit on a branch from this remote. * `git_name`: (optional) The name actually used for this git remote, if it's different from the default.
conf/remotes.yml +6 -3
@@ -1,6 +1,9 @@ torvalds: - commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id= + git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git + commit_url_suffix: /commit/?id= stable: - commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit?id= + git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git + commit_url_suffix: /commit?id= cip: - commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git/commit?id= + git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git + commit_url_suffix: /commit?id=
scripts/import_stable.py +9 -0
@@ -35,6 +35,9 @@ def update(git_repo, remote_name): subprocess.check_call(['git', 'remote', 'update', remote_name], cwd=git_repo) +def add(git_repo, remote_name, remote_url): + subprocess.check_call(['git', 'remote', 'add', remote_name, remote_url], + cwd=git_repo) def get_backports(git_repo, remotes, branches, debug=False): backports = {}
@@ -140,6 +143,12 @@ def main(git_repo, remotes, debug=False): remote_names = set(branch['git_remote'] for branch in branches) for remote_name in remote_names: + import sys + current_remotes = subprocess.check_output(['git', 'remote', 'show'], + cwd=git_repo).decode(sys.stdout.encoding).strip().split('\n') + if remote_name not in current_remotes: + add(git_repo, remotes[remote_name]['git_name'], + remotes[remote_name]['git_repo_url']) update(git_repo, remotes[remote_name]['git_name']) backports = get_backports(git_repo, remotes, branches, debug) c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)
scripts/templates/issue.html +4 -3
@@ -75,7 +75,8 @@ <th rowspan={{ branches|length }}>Status</th> {% for branch, affected in branches %} {% set name = branch.short_name %} - {% set url_prefix = remotes[branch.git_remote].commit_url_prefix %} + {% set git_repo_url = remotes[branch.git_remote].git_repo_url %} + {% set url_suffix = remotes[branch.git_remote].commit_url_suffix %} <th> <a href="/branch/{{ name }}/">{{ name }}</a> </th>
@@ -84,7 +85,7 @@ {% if issue['fixed-by'] and issue['fixed-by'][name] and issue['fixed-by'][name] != 'never' %} <span class="good">fixed</span> by {% for commit in issue['fixed-by'][name] %} - <a href="{{ url_prefix }}{{ commit }}">{{ commit[:12] }}</a>{% if not loop.last %},{% endif %} + <a href="{{ git_repo_url }}{{ url_suffix }}{{ commit }}">{{ commit[:12] }}</a>{% if not loop.last %},{% endif %} {% endfor %} {% else %} <span class="good">never affected</span>
@@ -98,7 +99,7 @@ {% if issue['introduced-by'] and issue['introduced-by'][name] and issue['introduced-by'][name] != 'never' %} - introduced by {% for commit in issue['introduced-by'][name] %} - <a href="{{ url_prefix }}{{ commit }}">{{ commit[:12] }}</a>{% if not loop.last %},{% endif %} + <a href="{{ git_repo_url }}{{ url_suffix }}{{ commit }}">{{ commit[:12] }}</a>{% if not loop.last %},{% endif %} {% endfor %} {% endif %} {% endif %}
Signature · 1 line
2.17.1
PATCH 2/2 git_repo: add a check for the git_repo directory
Users are required to prepare a local git repository, by default "../kernel", however there is no check at the moment. Alternatively, we could also create a repository for them (mkdir ../kernel; git init .) but that for now let's just spit an error.
--- scripts/import_stable.py | 1 + scripts/kernel_sec/branch.py | 11 +++++++++++ scripts/report_affected.py | 1 + scripts/webview.py | 1 + 4 files changed, 14 insertions(+)
scripts/import_stable.py +1 -0
@@ -179,6 +179,7 @@ if __name__ == '__main__': dest='git_repo', default='../kernel', help=('git repository from which to read commit logs ' '(default: ../kernel)'), + type=kernel_sec.branch.check_git_repo, metavar='DIRECTORY') parser.add_argument('--remote-name', dest='remote_name', action='append', default=[],
scripts/kernel_sec/branch.py +11 -0
@@ -219,3 +219,14 @@ def get_remotes(mappings, mainline=None, stable=None): if stable: remotes['stable']['git_name'] = stable return remotes + + +def check_git_repo(git_repo): + import argparse + if not os.path.isdir(git_repo): + msg = "directory %r not present" % git_repo + raise argparse.ArgumentTypeError(msg) + if not os.path.isdir(os.path.join(git_repo, '.git')): + msg = "directory %r is not a git repository" % git_repo + raise argparse.ArgumentTypeError(msg) + return git_repo
scripts/report_affected.py +1 -0
@@ -72,6 +72,7 @@ if __name__ == '__main__': dest='git_repo', default='../kernel', help=('git repository from which to read commit logs ' '(default: ../kernel)'), + type=kernel_sec.branch.check_git_repo, metavar='DIRECTORY') parser.add_argument('--remote-name', dest='remote_name', action='append', default=[],
scripts/webview.py +1 -0
@@ -202,6 +202,7 @@ if __name__ == '__main__': dest='git_repo', default='../kernel', help=('git repository from which to read commit logs ' '(default: ../kernel)'), + type=kernel_sec.branch.check_git_repo, metavar='DIRECTORY') parser.add_argument('--remote-name', dest='remote_name', action='append', default=[],
Signature · 1 line
2.17.1
PATCH 1/2 remotes: automatically add remotes from configuration file
In reply to Daniel SangorrinQuoted 5 lines · Daniel Sangorrin
Currently the user is required to create its own remotes by hand. This should not be necessary, because the information is already collected in
conf/remotes.yml. For that reason, if we detect that any remote has not been added to the local repo then we will add those.
I'm not yet convinced this is a good idea.
After this change, we would still be assuming that there is a kernel repository or working tree next to kernel-config. If we wanted to be really helpful we could create a new repository if it's not there.
But realistically, a developer using this probably already has a kernel git repository, just not in the expected place. Isn't it likely that the developer also already has the remotes we want, but under different names?
Aside from that, I have some specific issues with the implementation:
Quoted 16 lines
Signed-off-by Daniel Sangorrin <[email protected]>---
?README.md????????????????????| 5 +++-- ?conf/remotes.yml?????????????| 9 ++++++--- ?scripts/import_stable.py?????| 9 +++++++++ ?scripts/templates/issue.html | 7 ++++---?4 files changed, 22 insertions(+), 8 deletions(-)
README.md +0
-0@@ -77,8 +77,9 @@ These files, if they exist, contain a mapping where the keys ?are default git remote names.??The values are also mappings,?with the keys:
?
README.md These files, if they exist, contain a mapping where the keys
Quoted 5 lines
README.md +3
-2-* `commit_url_prefix`: URL prefix for browsing a commit on a -??branch from this remote. +* `git_repo_url`: URL of the remote repository. +* `commit_url_suffix`: URL suffix that gets appended to `git_repo_url` +??for browsing a commit on a branch from this remote.
Although it's now common to have a single https: URL that works as both a git remote and a web view, there are plenty of git hosts that don't work that way.
So these two things should be kept independent.
[...]
Quoted 5 lines
scripts/import_stable.py +0
-0@@ -35,6 +35,9 @@ def update(git_repo, remote_name): ?????subprocess.check_call(['git', 'remote', 'update', remote_name], ???????????????????????????cwd=git_repo)
Quoted 7 lines
?
+defadd(git_repo, remote_name, remote_url):
+????subprocess.check_call(['git', 'remote', 'add',remote_name,remote_url], +??????????????????????????cwd=git_repo)
?
?defget_backports(git_repo, remotes, branches, debug=False):
?????backports = {}
There should be two blank lines between top-level definitions, according to PEP-8.
scripts/import_stable.py def main(git_repo, remotes, debug=False): · line 146
Quoted 5 lines
scripts/import_stable.py +1
-0@@ -140,6 +143,12 @@ def main(git_repo, remotes, debug=False): remote_names = set(branch['git_remote'] for branch in branches) for remote_name in remote_names: + import sys
This import belongs at the top level with the other imports.
scripts/import_stable.py def main(git_repo, remotes, debug=False): · line 148
Quoted 2 lines
scripts/import_stable.py +2
-0+ current_remotes = subprocess.check_output(['git', 'remote', 'show'], + cwd=git_repo).decode(sys.stdout.encoding).strip().split('\n')
This doesn't belong inside the loop.
scripts/import_stable.py def main(git_repo, remotes, debug=False): · line 154
Quoted 6 lines
scripts/import_stable.py +3
-0+ if remote_name not in current_remotes: + add(git_repo, remotes[remote_name]['git_name'], + remotes[remote_name]['git_repo_url']) update(git_repo, remotes[remote_name]['git_name']) backports = get_backports(git_repo, remotes, branches, debug) c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)
Ben.
Signature · 3 lines
Ben Hutchings, Software Developer ? Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United KingdomPATCH 1/2 remotes: automatically add remotes from configuration file
Quoted 9 lines
From: Ben Hutchings <[email protected]>
Quoted 5 lines · Daniel Sangorrin
Currently the user is required to create its own remotes by hand. This should not be necessary, because the information is already collected in
conf/remotes.yml. For that reason, if we detect that any remote has not been added to the local repo then we will add those.I'm not yet convinced this is a good idea.
I sent a new patch that takes a different approach. The new patch will only check that the necessary remotes are there. I also prepared a new separate script, for new users to prepare the local repository.
Quoted 3 lines
After this change, we would still be assuming that there is a kernel repository or working tree next to kernel-config. If we wanted to be really helpful we could create a new repository if it's not there.
I didn't understand the kernel-config thing. Maybe I am missing something important.
Quoted 6 lines
But realistically, a developer using this probably already has a kernel git repository, just not in the expected place. Isn't it likely that the developer also already has the remotes we want, but under different names?
Aside from that, I have some specific issues with the implementation:
Thanks the new patch has taken those issues into account.
Kind regards,
Daniel
Quoted 72 lines
Quoted 17 lines
Signed-off-by Daniel Sangorrin <[email protected]>---
?README.md????????????????????| 5 +++-- ?conf/remotes.yml?????????????| 9 ++++++--- ?scripts/import_stable.py?????| 9 +++++++++ ?scripts/templates/issue.html | 7 ++++---?4 files changed, 22 insertions(+), 8 deletions(-)
README.md +0
-0@@ -77,8 +77,9 @@ These files, if they exist, contain a mapping where the keys ?are default git remote names.??The values are also mappings,?with the keys:
README.md These files, if they exist, contain a mapping where the keys
Quoted 5 lines
README.md +3
-2-* `commit_url_prefix`: URL prefix for browsing a commit on a -??branch from this remote. +* `git_repo_url`: URL of the remote repository. +* `commit_url_suffix`: URL suffix that gets appended to `git_repo_url` +??for browsing a commit on a branch from this remote.Although it's now common to have a single https: URL that works as both a git remote and a web view, there are plenty of git hosts that don't work that way.
So these two things should be kept independent.
[...]
Quoted 10 lines
scripts/import_stable.py +3
-0@@ -35,6 +35,9 @@ def update(git_repo, remote_name): ?????subprocess.check_call(['git', 'remote', 'update', remote_name], ???????????????????????????cwd=git_repo) +def add(git_repo, remote_name, remote_url): +????subprocess.check_call(['git', 'remote', 'add', remote_name, remote_url], +??????????????????????????cwd=git_repo)Quoted 2 lines
?def
get_backports(git_repo, remotes, branches, debug=False):
?????backports = {}There should be two blank lines between top-level definitions, according to PEP-8.
scripts/import_stable.py def main(git_repo, remotes, debug=False): · line 146
Quoted 5 lines
scripts/import_stable.py +1
-0@@ -140,6 +143,12 @@ def main(git_repo, remotes, debug=False): remote_names = set(branch['git_remote'] for branch in branches) for remote_name in remote_names: + import sysThis import belongs at the top level with the other imports.
scripts/import_stable.py def main(git_repo, remotes, debug=False): · line 148
Quoted 2 lines
scripts/import_stable.py +2
-0+ current_remotes = subprocess.check_output(['git', 'remote', 'show'], + cwd=git_repo).decode(sys.stdout.encoding).strip().split('\n')This doesn't belong inside the loop.
scripts/import_stable.py def main(git_repo, remotes, debug=False): · line 154
Quoted 6 lines
scripts/import_stable.py +3
-0+ if remote_name not in current_remotes: + add(git_repo, remotes[remote_name]['git_name'], + remotes[remote_name]['git_repo_url']) update(git_repo, remotes[remote_name]['git_name']) backports = get_backports(git_repo, remotes, branches, debug) c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)Ben.
--
Ben Hutchings, Software Developer ? Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United Kingdom
PATCH 1/2 remotes: automatically add remotes from configuration file
Quoted 14 lines · daniel.sangorrin at toshiba.co.jp
Quoted 9 lines
Quoted 1 line
Quoted 1 line
From: Ben Hutchings <[email protected]>
Quoted 5 lines · Daniel Sangorrin
Currently the user is required to create its own remotes by hand. This should not be necessary, because the information is already collected in
conf/remotes.yml. For that reason, if we detect that any remote has not been added to the local repo then we will add those.I'm not yet convinced this is a good idea.
I sent a new patch that takes a different approach. The new patch will only check that the necessary remotes are there. I also prepared a new separate script, for new users to prepare the local repository.
Quoted 5 lines
Quoted 3 lines
After this change, we would still be assuming that there is a kernel repository or working tree next to kernel-config.??If we wanted to be really helpful we could create a new repository if it's not there.
I didn't understand the kernel-config thing. Maybe I am missing something important.
[...]
Sorry, I meant "next to kernel-sec".
Ben.
Signature · 3 lines
Ben Hutchings, Software Developer ? Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United Kingdom