backports: add Python based get-backport-kernels
PATCH v2 1/1 backports: add Python based get-backport-kernels
This replaces the old shell get-compat-kernels with a shiny Python based get-backport-kernels. The new version provides a few features which were simply a difficult to implement and quite frankly pointless to try to implement in shell. Things worth mentioning:
* This will now monitor your old directories and packages and remove any stale old directory or packages. You should feel comfortable with now just getting the latest code from the repo and always just running the script, it will do everything you wished it did for you.
* This will always check the URLs and ensure your file size matches the target upstream size, we used to only check if you had the file or not
* The compat-ksrc directory was renamed to
ksrc-backports which goes inline with our
rebranding crusade. The tool is smart enough
to figure out if you had the old name and
rename it for you. Its also smart enough to
figure out if you had your old compat-ksrc
as a symlink and preserve things just as you
intended following the old real path.
* The debs/ directory is now moved into
ksrc-backports, making it easier to manage
and detect what is or not part of backports.
* I found no way to deal with objects to extract GNU archive items with Python, so we're stuck with requiring GNU ar, this provides a simple wrapper for our usage.
Relying on an continued update list is dumb though, in the future we should look into putting out new builds through without requiring GNU ar and also updating the list through a json file just as the kernel has its own now:
https://www.kernel.org/releases.json
---
This v2 also contains a 1 line required change to ckmake.
devel/ckmake | 2 +- devel/get-backport-kernels | 523 +++++++++++++++++++++++++++++++++++++++++++++ devel/get-compat-kernels | 268 ----------------------- lib/bpar.py | 17 ++ 4 files changed, 541 insertions(+), 269 deletions(-) create mode 100755 devel/get-backport-kernels delete mode 100755 devel/get-compat-kernels create mode 100644 lib/bpar.py
devel/ckmake +1 -1
@@ -37,7 +37,7 @@ ckmake_log = my_cwd + "/ckmake.log" ckmake_report = my_cwd + "/ckmake-report.log" home = os.getenv("HOME") -ksrc = home + "/" + "compat-ksrc" +ksrc = home + "/" + "ksrc-backports" modules = ksrc + "/lib/modules" def clean():
devel/get-backport-kernels new file +523 -0
523 lines not on this page · raw message
devel/get-compat-kernels deleted +0 -268
268 lines not on this page · raw message
lib/bpar.py new file +17 -0
@@ -0,0 +1,17 @@ +import subprocess, os + +class ArError(Exception): + pass +class ExecutionError(ArError): + def __init__(self, errcode): + self.error_code = errcode + +def print_data(input_file, out_file, tree=None): + cmd = ['ar', 'p', input_file, 'data.tar.gz'] + process = subprocess.Popen(cmd, + stdout=out_file, stderr=subprocess.STDOUT, + close_fds=True, universal_newlines=True, cwd=tree) + stdout = process.communicate()[0] + process.wait() + if process.returncode != 0: + raise ExecutionError(process.returncode)
Signature · 1 line
1.8.4.3
Quoted 45 lines · Luis R. Rodriguez
This replaces the old shell get-compat-kernels with a shiny Python based get-backport-kernels. The new version provides a few features which were simply a difficult to implement and quite frankly pointless to try to implement in shell. Things worth mentioning:
* This will now monitor your old directories and packages and remove any stale old directory or packages. You should feel comfortable with now just getting the latest code from the repo and always just running the script, it will do everything you wished it did for you.
* This will always check the URLs and ensure your file size matches the target upstream size, we used to only check if you had the file or not
* The compat-ksrc directory was renamed to ksrc-backports which goes inline with our rebranding crusade. The tool is smart enough to figure out if you had the old name and rename it for you. Its also smart enough to figure out if you had your old compat-ksrc as a symlink and preserve things just as you intended following the old real path.
* The debs/ directory is now moved into ksrc-backports, making it easier to manage and detect what is or not part of backports.
* I found no way to deal with objects to extract GNU archive items with Python, so we're stuck with requiring GNU ar, this provides a simple wrapper for our usage.
Relying on an continued update list is dumb though, in the future we should look into putting out new builds through without requiring GNU ar and also updating the list through a json file just as the kernel has its own now:
https://www.kernel.org/releases.json
Signed-off-by Luis R. Rodriguez <[email protected]>
Quoted 8 lines
---
This v2 also contains a 1 line required change to ckmake.
devel/ckmake | 2 +- devel/get-backport-kernels | 523 +++++++++++++++++++++++++++++++++++++++++++++ devel/get-compat-kernels | 268 ----------------------- lib/bpar.py | 17 ++
I just tried this on the compat server, but I am getting some errors:
./devel/get-backport-kernels
......
3.5.7 - installing usr/src/ generic solution ... 3.8.13 - installing usr/src/ ...
3.8.13 - rebuilding binary dependencies ...
CLEAN scripts/basic
HOSTCC scripts/basic/fixdep
Building modules, stage 2.
MODPOST 0 modules
CLEAN scripts/mod
CHECK scripts/mod/empty.c/bin/sh: 1: sparse: not found
make[1]: *** [scripts/mod/empty.o] Error 127 make[1]: *** Waiting for unfinished jobs....
HOSTCC scripts/mod/mk_elfconfig
make: *** [_module_scripts/mod] Error 2
CLEAN scripts/genksyms
HOSTCC scripts/genksyms/genksyms.o SHIPPED scripts/genksyms/lex.lex.c SHIPPED scripts/genksyms/keywords.hash.c SHIPPED scripts/genksyms/parse.tab.h SHIPPED scripts/genksyms/parse.tab.c
HOSTCC scripts/genksyms/lex.lex.o HOSTCC scripts/genksyms/parse.tab.o HOSTLD scripts/genksyms/genksyms
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
CHECK scripts/mod/empty.c/bin/sh: 1: sparse: not found make[1]: *** [scripts/mod/empty.o] Error 127
make: *** [_module_scripts/mod] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
CHECK scripts/mod/empty.c/bin/sh: 1: sparse: not found make[1]: *** [scripts/mod/empty.o] Error 127 make[1]: *** Waiting for unfinished jobs....
make: *** [_module_scripts/mod] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
CHECK scripts/mod/empty.c/bin/sh: 1: sparse: not found make[1]: *** [scripts/mod/empty.o] Error 127
make: *** [_module_scripts/mod] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
3.5.7 - rebuilding binary dependencies ...
CLEAN scripts/basic
HOSTCC scripts/basic/fixdep Building modules, stage 2.
MODPOST 0 modules
CLEAN scripts/mod
CHECK scripts/mod/empty.c/bin/sh: 1: sparse: not found
HOSTCC scripts/mod/mk_elfconfig
make[1]: *** [scripts/mod/empty.o] Error 127 make[1]: *** Waiting for unfinished jobs....
make: *** [_module_scripts/mod] Error 2
CLEAN scripts/genksyms
HOSTCC scripts/genksyms/genksyms.o SHIPPED scripts/genksyms/lex.lex.c SHIPPED scripts/genksyms/keywords.hash.c SHIPPED scripts/genksyms/parse.tab.h SHIPPED scripts/genksyms/parse.tab.c
HOSTCC scripts/genksyms/lex.lex.o HOSTCC scripts/genksyms/parse.tab.o HOSTLD scripts/genksyms/genksyms
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
CHECK scripts/mod/empty.c/bin/sh: 1: sparse: not found make[1]: *** [scripts/mod/empty.o] Error 127
make: *** [_module_scripts/mod] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
CHECK scripts/mod/empty.c/bin/sh: 1: sparse: not found make[1]: *** [scripts/mod/empty.o] Error 127
make: *** [_module_scripts/mod] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
CHECK scripts/mod/empty.c/bin/sh: 1: sparse: not found make[1]: *** [scripts/mod/empty.o] Error 127 make[1]: *** Waiting for unfinished jobs....
make: *** [_module_scripts/mod] Error 2
Building modules, stage 2.
MODPOST 0 modules/bin/sh: 1: scripts/mod/modpost: not found make[1]: *** [__modpost] Error 127
make: *** [modules] Error 2
Quoted 1 line · Hauke Mehrtens
sparse: not found
Doh, I made a huge typo and used C= instead of -C :) I'll submit a v3 next.
Luis