q-rig: qemu ltprun on armhf and arm64
Hi all,
this is me, Domenico (cavokz on github.com, cavok at debian.org).
I've been toying with Travis CI, qemu and LTP recently. My effort is here: https://github.com/q-rig. It runs LTP on top of the latest linux -rc on Debian sid, inside qemu on armhf and arm64. Binaries are built when needed. You can find them here:
https://github.com/q-rig/qemu-ltprun-results.
Activities are run in Travis CI [0]. LTP tests are split so to finish in less than 49 minutes (Travis CI constraint). The LPT results are put again in https://github.com/q-rig/qemu-ltprun-results, which is probably the wrong place.
All this is quite brute force cpu cycles harvesting generously proved by Travis CI and github.
Now I'd like to tackle the aggregation of the data generated by LTP. The best I could do is just generating also the .html page and upload it to github [1] but I think more is actually desirable here.
Is there any documentation I'm expected to read before asking for support here? :)
Thanks,
Domenico
[0] https://travis-ci.org/q-rig/qemu-ltprun
[1] https://github.com/q-rig/qemu-ltprun-results/tree/master/ltp/20170411-3.10-linux-4.11.0-rc6-ltp-20170116-arm64
Signature · 1 line
3B10 0CA1 8674 ACBA B4FE FCD2 CE5B CF17 9960 DE13
Hi!
Quoted 10 lines
I've been toying with Travis CI, qemu and LTP recently. My effort is here: https://github.com/q-rig. It runs LTP on top of the latest linux -rc on Debian sid, inside qemu on armhf and arm64. Binaries are built when needed. You can find them here:
https://github.com/q-rig/qemu-ltprun-results.Activities are run in Travis CI [0]. LTP tests are split so to finish in less than 49 minutes (Travis CI constraint). The LPT results are put again in https://github.com/q-rig/qemu-ltprun-results, which is probably the wrong place.
Interesting work, although I'm not sure that travis is right tool for the job. On the other hand I've been thinking of similar misuse as well...
Quoted 3 lines
Now I'd like to tackle the aggregation of the data generated by LTP. The best I could do is just generating also the .html page and upload it to github [1] but I think more is actually desirable here.
I would like to be able to do more with the LTP results as well, for instance compare two testruns for the purpose of automatic pre-release testing which would allow us to do much shorter release cycles.
Unfortunately the best the legacy LTP runltp script + ltp-pan can do is to write a list of failed and skipped tests, it also can't run the open posix testcases, cannot kill tests on timeout, etc.
This are the reasons I've started to experiment with writing a new test runner that would allow for writing different output formats among others some kind of machine parseable output (I was thinking of json but anything structured would do) which would allow us to produce much more interesting output than the current html page. Proof of concet is done in my private GitHub repo[1], but it's not complete enough to be usable and it can produce only html[2] page as an output so far, but hacking in json formatter would be fairly easy.
[1] https://github.com/metan-ucw/ltp
https://github.com/metan-ucw/ltp/tree/master/tools/runltp-ng
[2] http://metan.ucw.cz/outgoing/ltp-openposix.html
http://metan.ucw.cz/outgoing/ltp-syscalls.html
The latest version also includes hardware and sofware version but apart from that it's more or less it.
Quoted 2 lines
Is there any documentation I'm expected to read before asking for support here? :)
There is probably none or close to that hence asking here is the best course of action.
Signature · 2 lines
Cyril Hrubis [email protected]