patches to check remotes
Hello Ben,
Please consider these patches for cip-kernel-sec:
[cip-kernel-sec 1/3] remotes: check that remotes exist
This patch adds checks on the local repository and also provides a separate script to prepare it. Expert kernel maintainers will only need the checks, but for new users the script might be useful.
*Important* this patch deprecates the patch "git_repo: add a check for the git_repo directory" You haven't reviewed or accepted that patch as far as I know. You can safely ignore it, because this one provides better checking and takes into account remotes mapping.
[cip-kernel-sec 2/3] branch: fix remotes mapping
This was a bug in the remote mapping code.
[cip-kernel-sec 3/3] help:remote_name: use colon instead of equal
This is just a fix to command's help
Thanks,
Daniel
PATCH 1/3 remotes: check that remotes exist
This patch introduces some checks for the local git repository containing the remotes configured in conf/remotes.yml
These checks occur at the beginning of import_stable.py, report_affected.py and webview.py, to make sure that the local repository is consistent with the configuration yaml files.
We also add a new script, prepare_remotes.py, that is useful to bootstrap the local git repository with the configured remote branches. Expert users may want to use their own, but the script is helpful for new users or a quickstart.
Both the checks and the new script, take repository name mappings into account.
--- README.md | 5 ++++ conf/remotes.yml | 3 +++ scripts/import_stable.py | 9 +++---- scripts/kernel_sec/branch.py | 42 +++++++++++++++++++++++++++++++ scripts/prepare_remotes.py | 49 ++++++++++++++++++++++++++++++++++++ scripts/report_affected.py | 1 + scripts/webview.py | 1 + 7 files changed, 104 insertions(+), 6 deletions(-) create mode 100755 scripts/prepare_remotes.py
README.md +5 -0
@@ -23,6 +23,10 @@ this is assumed to be in `../kernel`, with remotes configured in stable and cip repositories. These can be overridden by command-line options or configuration (`~/.config/kernel-sec/remotes.yml`). +* `scripts/prepare_remotes.py` - creates the local git repository +and adds all configured remotes. You may prefer to skip this script +and configure the repository by hand. + * `scripts/import_debian.py` - import information from Debian's `kernel_sec` project. It includes all issues that Debian considers active or that are already tracked here.
@@ -81,6 +85,7 @@ with the keys: branch from this remote. * `git_name`: (optional) The name actually used for this git remote, if it's different from the default. +* `git_repo_url`: URL of the remote git repository. ## Contributions
conf/remotes.yml +3 -0
@@ -1,6 +1,9 @@ torvalds: commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id= + git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git stable: commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit?id= + git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git cip: commit_url_prefix: https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git/commit?id= + git_repo_url: https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git
scripts/import_stable.py +3 -6
@@ -31,11 +31,6 @@ BACKPORT_COMMIT_BOTTOM_RE = re.compile( .format(**RE_USE)) -def update(git_repo, remote_name): - subprocess.check_call(['git', 'remote', 'update', remote_name], - cwd=git_repo) - - def get_backports(git_repo, remotes, branches, debug=False): backports = {}
@@ -140,7 +135,8 @@ def main(git_repo, remotes, debug=False): remote_names = set(branch['git_remote'] for branch in branches) for remote_name in remote_names: - update(git_repo, remotes[remote_name]['git_name']) + kernel_sec.branch.remote_update( + git_repo, remotes[remote_name]['git_name']) backports = get_backports(git_repo, remotes, branches, debug) c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)
@@ -190,4 +186,5 @@ if __name__ == '__main__': remotes = kernel_sec.branch.get_remotes(args.remote_name, mainline=args.mainline_remote_name, stable=args.stable_remote_name) + kernel_sec.branch.check_git_repo(args.git_repo, remotes) main(args.git_repo, remotes, args.debug)
scripts/kernel_sec/branch.py +42 -0
55 lines not on this page · raw message
scripts/prepare_remotes.py new file +49 -0
49 lines not on this page · raw message
scripts/report_affected.py +1 -0
@@ -100,5 +100,6 @@ if __name__ == '__main__': remotes = kernel_sec.branch.get_remotes(args.remote_name, mainline=args.mainline_remote_name, stable=args.stable_remote_name) + kernel_sec.branch.check_git_repo(args.git_repo, remotes) main(args.git_repo, remotes, args.only_fixed_upstream, args.include_ignored, *args.branches)
scripts/webview.py +1 -0
@@ -219,6 +219,7 @@ if __name__ == '__main__': remotes = kernel_sec.branch.get_remotes(args.remote_name, mainline=args.mainline_remote_name, stable=args.stable_remote_name) + kernel_sec.branch.check_git_repo(git_repo, remotes) conf = { '/static/style.css': {
Signature · 1 line
2.17.1
PATCH 1/3 remotes: check that remotes exist
This mostly looks fine, except:
On Tue, 2019-06-18 at 11:40 +0900, Daniel Sangorrin wrote:
[...]
Quoted 2 lines
scripts/prepare_remotes.py new file +0
-0
[...]
Quoted 2 lines
+def
main(git_repo, remotes, debug=False):
+????kernel_sec.branch.check_git_repo(git_repo, remotes, create=True)
The debug flag doesn't have any effect...
[...]
Quoted 3 lines
+????parser.add_argument('--debug', +????????????????????????dest='debug', action='store_true', +????????????????????????help='enable debugging output')
[...]
...so neither does the --debug option.
I think you can remove them both.
Ben.
Signature · 3 lines
Ben Hutchings, Software Developer ? Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United KingdomPATCH 1/3 remotes: check that remotes exist
Hello Ben,
Quoted 37 lines
-----Original Message-----
From: Ben Hutchings <[email protected]>
Sent: Wednesday, June 19, 2019 12:07 AM
To: sangorrin daniel(????? ???? ????????) <[email protected]>
Cc: cip-dev at lists.cip-project.org
Subject: Re: [cip-kernel-sec 1/3] remotes: check that remotes existQuoted 28 lines
-----Original Message-----
From: Ben Hutchings <[email protected]>
Sent: Wednesday, June 19, 2019 12:07 AM
To: sangorrin daniel(????? ???? ????????) <[email protected]>
Cc: cip-dev at lists.cip-project.org
Subject: Re: [cip-kernel-sec 1/3] remotes: check that remotes existThis mostly looks fine, except:
On Tue, 2019-06-18 at 11:40 +0900, Daniel Sangorrin wrote:
[...]Quoted 2 lines
scripts/prepare_remotes.py new file +0
-0[...]
Quoted 2 lines
+def
main(git_repo, remotes, debug=False):
+????kernel_sec.branch.check_git_repo(git_repo, remotes, create=True)The debug flag doesn't have any effect...
[...]
Quoted 3 lines
+????parser.add_argument('--debug', +????????????????????????dest='debug', action='store_true', +????????????????????????help='enable debugging output')
[...]
...so neither does the --debug option.
I think you can remove them both.
Thanks.
I have sent you a 2nd version of the 3 patch series.
Kind regards,
Daniel
Quoted 7 lines
Quoted 7 lines
Ben.
--
Ben Hutchings, Software Developer ? Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United Kingdom
PATCH 2/3 branch: fix remotes mapping
In reply to Daniel SangorrinProbably a copy&paste error that never got checked because nobody is using the mapping functionality.
--- scripts/kernel_sec/branch.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/kernel_sec/branch.py +1 -1
@@ -214,7 +214,7 @@ def get_remotes(mappings, mainline=None, stable=None): _get_configured_remotes( os.path.expanduser('~/.config/kernel-sec/remotes.yml'))) for mapping in mappings: - left, right = arg.split(':', 1) + left, right = mapping.split(':', 1) remotes[left]['git_name'] = right if mainline: remotes['torvalds']['git_name'] = mainline
Signature · 1 line
2.17.1
PATCH 2/3 branch: fix remotes mapping
Quoted 2 lines · Daniel Sangorrin
Probably a copy&paste error that never got checked because nobody is using the mapping functionality.
I thought I'd tested this. :-(
Ben.
Quoted 18 lines
Signed-off-by Daniel Sangorrin <[email protected]>---
?scripts/kernel_sec/branch.py | 2 +-
?1 file changed, 1 insertion(+), 1 deletion(-)scripts/kernel_sec/branch.py +0
-0@@ -214,7 +214,7 @@ def get_remotes(mappings, mainline=None, stable=None):?????????_get_configured_remotes( ?????????????os.path.expanduser('~/.config/kernel-sec/remotes.yml'))) ?????for mapping in mappings:
-????????left, right =arg.split(':', 1)
+????????left, right =mapping.split(':', 1)?????????remotes[left]['git_name'] = right
?????if mainline:
?????????remotes['torvalds']['git_name'] = mainline
Signature · 3 lines
Ben Hutchings, Software Developer ? Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United KingdomPATCH 3/3 help:remote_name: use colon instead of equal
In reply to Daniel SangorrinThe example and implementation use a colon (:) to specify remote mappings, but the command's help use the equal sign (=).
--- scripts/import_stable.py | 2 +- scripts/prepare_remotes.py | 2 +- scripts/report_affected.py | 2 +- scripts/webview.py | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-)
scripts/import_stable.py +1 -1
@@ -170,7 +170,7 @@ if __name__ == '__main__': parser.add_argument('--remote-name', dest='remote_name', action='append', default=[], help='git remote name mappings, e.g. stable:korg-stable', - metavar='NAME=OTHER-NAME') + metavar='NAME:OTHER-NAME') parser.add_argument('--mainline-remote', dest='mainline_remote_name', help="git remote name to use instead of 'torvalds'",
scripts/prepare_remotes.py +1 -1
@@ -30,7 +30,7 @@ if __name__ == '__main__': parser.add_argument('--remote-name', dest='remote_name', action='append', default=[], help='git remote name mappings, e.g. stable:korg-stable', - metavar='NAME=OTHER-NAME') + metavar='NAME:OTHER-NAME') parser.add_argument('--mainline-remote', dest='mainline_remote_name', help="git remote name to use instead of 'torvalds'",
scripts/report_affected.py +1 -1
@@ -76,7 +76,7 @@ if __name__ == '__main__': parser.add_argument('--remote-name', dest='remote_name', action='append', default=[], help='git remote name mappings, e.g. stable:korg-stable', - metavar='NAME=OTHER-NAME') + metavar='NAME:OTHER-NAME') parser.add_argument('--mainline-remote', dest='mainline_remote_name', help="git remote name to use instead of 'torvalds'",
scripts/webview.py +1 -1
@@ -206,7 +206,7 @@ if __name__ == '__main__': parser.add_argument('--remote-name', dest='remote_name', action='append', default=[], help='git remote name mappings, e.g. stable:korg-stable', - metavar='NAME=OTHER-NAME') + metavar='NAME:OTHER-NAME') parser.add_argument('--mainline-remote', dest='mainline_remote_name', help="git remote name to use instead of 'torvalds'",
Signature · 1 line
2.17.1
PATCH 3/3 help:remote_name: use colon instead of equal
Quoted 5 lines · Daniel Sangorrin
The example and implementation use a colon (:) to specify remote mappings, but the command's help use the equal sign (=).
Signed-off-by Daniel Sangorrin <[email protected]>
[...]
Right.
Ben.
Signature · 3 lines
Ben Hutchings, Software Developer ? Codethink Ltd
https://www.codethink.co.uk/ Dale House, 35 Dale Street
Manchester, M1 2HF, United Kingdom