perl/Git.pm: Writing a test suite
Git.pm doesn't currently have a test suite, and I think it should get one.
Since Git.pm makes calls to the git binary, it would seem sensible to me to create mock output for the external calls, like Grit (a Ruby API) does it, here: http://github.com/mojombo/grit/tree/master/test/fixtures
(Having a complete test repository to test against like Ruby/Git
(another API) at http://repo.or.cz/w/rubygit.git?a=tree;f=tests/files;hb=HEAD is probably too brittle and makes it hard to test edge cases.)
I haven't done testing in perl before, so if someone wants to exchange ideas and/or help me get this set up, please post here or message me on IRC (lea_w/lea_1 on #git at Freenode).
-- Lea
Hi,
Quoted 8 lines · Lea Wiemann
Git.pm doesn't currently have a test suite, and I think it should get one.
Since Git.pm makes calls to the git binary, it would seem sensible to me to create mock output for the external calls, like Grit (a Ruby API) does it,
here: http://github.com/mojombo/grit/tree/master/test/fixtures (Having a complete test repository to test against like Ruby/Git (another API) at
http://repo.or.cz/w/rubygit.git?a=tree;f=tests/files;hb=HEAD is probably too brittle and makes it hard to test edge cases.)
I think this is bad idea; you should test not just Git.pm
functionality, but equally importantly also its integration with (current version of) Git. If Git's output/behaviour changes in some subtle way, you want to see if it breaks Git.pm.
Quoted 3 lines
I haven't done testing in perl before, so if someone wants to exchange ideas and/or help me get this set up, please post here or message me on IRC (lea_w/lea_1 on #git at Freenode).
There are two possible ways, either using the Git test suite
infrastructure, or the Perl test suite infrastructure. I'm ambivalent about which one to choose, both have their advantages; I wonder what others' opinion is. I would personally go with the Perl test suite in this case but can't support it with much more than just a gut feeling.
Signature · 3 lines
Petr "Pasky" Baudis Whatever you can do, or dream you can, begin it. Boldness has genius, power, and magic in it. -- J. W. von Goethe
Quoted 6 lines · Petr Baudis
Quoted 1 line · Lea Wiemann
it would seem sensible to me to create mock output for the external calls
I think this is bad idea; you should test not just Git.pm
functionality, but equally importantly also its integration with (current version of) Git.
Good point. So I'll work on test repositories with real calls to Git where possible.
Quoted 2 lines
There are two possible ways, either using the Git test suite
infrastructure, or the Perl test suite infrastructure.
I'd definitely want to use Perl's testing infrastructure, but probably it's sensible to use both Git's and Perl's infrastructure (e.g. call the Git.pm perl tests from the Git test suite shell scripts, and maybe have setup code in the shell scripts). Here are two questions:
1. Is it OK for the Git.pm tests to be called from the Git test scripts but generate different-looking output (since the tests are run in Perl)?
2. I might need external test helpers from CPAN at some point. Is that a bad idea because of dependencies, or are we OK with having additional dependencies for running the test suite if necessary?
-- Lea
Quoted 3 lines · Lea Wiemann
1. Is it OK for the Git.pm tests to be called from the Git test scripts but generate different-looking output (since the tests are run in Perl)?
Cosmetics. As long as they are not disproportionally chatty nor too terse, and correctly catch breakages and stop (if told to with -i) on the first one, I think it is Ok to be different (e.g. no support for colored output).
Quoted 3 lines
2. I might need external test helpers from CPAN at some point. Is that a bad idea because of dependencies, or are we OK with having additional dependencies for running the test suite if necessary?
As long as you use the ones that is widely used in Perl world I do not think it would be a problem.
2008/5/30 Lea Wiemann <[email protected]>:
Quoted 9 lines
Git.pm doesn't currently have a test suite, and I think it should get one.
Since Git.pm makes calls to the git binary, it would seem sensible to me to create mock output for the external calls, like Grit (a Ruby API) does it,
here: http://github.com/mojombo/grit/tree/master/test/fixtures (Having a complete test repository to test against like Ruby/Git (another API) at
http://repo.or.cz/w/rubygit.git?a=tree;f=tests/files;hb=HEAD is probably too brittle and makes it hard to test edge cases.)
Quoted 3 lines
I haven't done testing in perl before, so if someone wants to exchange ideas and/or help me get this set up, please post here or message me on IRC (lea_w/lea_1 on #git at Freenode).
Testing is big in the perl world. I suggest to start with reading :
http://www.perlfoundation.org/perl5/index.cgi?testing
I'm not overloaded with tuits at the moment, but I'll try to review a bit your testing patches when you post them here.