Testing
Hi all,
I'd like to open some discussion about testing. I honestly haven't given that much thought to big picture testing but I can describe what I'm doing to btbridged to at least have something. I'm sure everyone has ideas as to which direction the testing should be going and what we should be doing and this is the conversation I would like to start.
At the moment it is almost impossible to test btbridged, short of actually booting a real openbmc and making the host to something and through observation try to see that all went well there really isn't a way of knowing if changes work.
Joel kindly added a .travis.yml (travis-ci.org can automate) file to the project to at least have compile checks, which is a great start. I'm currently expanding on it so include a 'fake' bt-host and a trivial ipmi client. I don't expect these tests to get too fancy but I do hope they'll be able to catch broken commits. Future direction might be to have everything done in qemu so that the 'fake' environment is closer to the real thing, doing so in qemu would also allow us to test the bt-host driver in linux.
Cyril
Quoted 5 lines · Cyril Bur
I'd like to open some discussion about testing. I honestly haven't given that much thought to big picture testing but I can describe what I'm doing to btbridged to at least have something. I'm sure everyone has ideas as to which direction the testing should be going and what we should be doing and this is the conversation I would like to start.
There's a number of different levels of things too. I know Chris has been fairly involved in testing the end user REST API with robotest and the like, although what you're mostly talking about here is unit tests and other such smaller box tests.
Quoted 4 lines
At the moment it is almost impossible to test btbridged, short of actually booting a real openbmc and making the host to something and through observation try to see that all went well there really isn't a way of knowing if changes work.
funnily enough, we have that problem with skiboot too. Especially for hardware that doesn't physically exist yet.
Quoted 7 lines
Joel kindly added a .travis.yml (travis-ci.org can automate) file to the project to at least have compile checks, which is a great start. I'm currently expanding on it so include a 'fake' bt-host and a trivial ipmi client. I don't expect these tests to get too fancy but I do hope they'll be able to catch broken commits. Future direction might be to have everything done in qemu so that the 'fake' environment is closer to the real thing, doing so in qemu would also allow us to test the bt-host driver in linux.
Good idea.
btw, if you adopted autotools, you could get parallel test running for free :)