New TKO: Test results comparison
PATCH 1/1 New TKO: Test results comparison
The general idea of test case comparison is to compare a given test case execution with other executions of the same test according to a predefined
list of attributes.
The comparison result is a list of test executions ordered by attributes mismatching distance (same idea as hamming distance, just implemented on test attributes). Distance of 0 refers to test executions that have identical attributes values, distance of 1 to test executions that have only one different attribute value etc.
As for the patch code, I have added the new 'Test case comparison' element under the Test Attributes element in the Test Details tab. Opening this element queries the DB and outputs a list of compared tests. I used the new_tko and django database model infrastructure and added a table to the tko database - test_comparison_attributes - that stores a list of attributes for comparison per test name per user.
The general layout of the comparison analysis result is:
differing attributes (nSuccess/nTotal)
attribute value (nSuccess/nTotal)
Status
testID reason
testID reason
...
Status (different)
testID reason
...A basic usage example can be viewed here:
http://picasaweb.google.com/qalogic.pub
The patches were re-worked against the latest SVN trunk and all the TKO database queries are made now using the Django database model.
--- frontend/client/src/autotest/public/TkoClient.html | 6 + frontend/client/src/autotest/public/tkoclient.css | 9 + .../client/src/autotest/tko/TestDetailView.java | 184 +++++++++- global_config.ini | 1 + new_tko/tko/models.py | 11 + new_tko/tko/rpc_interface.py | 368 +++++++++++++++++++- .../030_add_test_comparison_attributes.py | 20 + 7 files changed, 579 insertions(+), 20 deletions(-) create mode 100644 tko/migrations/030_add_test_comparison_attributes.py
frontend/client/src/autotest/public/TkoClient.html +6 -0
@@ -84,6 +84,11 @@ <div id="test_detail_view" title="Test details"> <span id="td_fetch" class="box-full">Fetch test by ID: <span id="td_fetch_controls"></span> + </span><br> + <span id="td_comp_attr" class="box-full"> + Compare this execution using attributes <span id="td_comp_attr_control"></span> + with previous executions of <span id="td_comp_name_control"></span> + <span id="td_comp_save_control"></span> </span><br><br> <div id="td_title" class="title"></div><br> <div id="td_data">
@@ -110,6 +115,7 @@ <span class="field-name">Test labels:</span> <span id="td_test_labels"></span><br> <span id="td_attributes"></span><br> + <span id="td_analysis"></span><br> <br> <span class="field-name">Key log files:</span>
frontend/client/src/autotest/public/tkoclient.css +9 -0
@@ -62,3 +62,12 @@ div.spreadsheet-cell-nonclickable { font-weight: bold; } +.test-analysis .content { + white-space: pre; + font-family: monospace; +} + +.test-analysis .header table { + font-weight: bold; +} +
frontend/client/src/autotest/tko/TestDetailView.java +165 -19
314 lines not on this page · raw message
global_config.ini +1 -0
@@ -11,6 +11,7 @@ query_timeout: 3600 min_retry_delay: 20 max_retry_delay: 60 graph_cache_creation_timeout_minutes: 10 +test_comparison_maximum_attribute_mismatches: 4 [AUTOTEST_WEB] host: localhost
new_tko/tko/models.py +11 -0
@@ -228,6 +228,17 @@ class IterationAttribute(dbmodels.Model, model_logic.ModelExtensions): db_table = 'iteration_attributes' +class TestComparisonAttribute(dbmodels.Model, model_logic.ModelExtensions): + testname = dbmodels.CharField(max_length=90) + # TODO: change this to foreign key once DBs are merged + owner = dbmodels.CharField(max_length=80) + attributes = dbmodels.CharField(max_length=1024) + testset = dbmodels.CharField(max_length=1024) + + class Meta: + db_table = 'test_comparison_attributes' + + class IterationResult(dbmodels.Model, model_logic.ModelExtensions): # see comment on IterationAttribute regarding primary_key=True test = dbmodels.ForeignKey(Test, db_column='test_idx', primary_key=True)
new_tko/tko/rpc_interface.py +367 -1
386 lines not on this page · raw message
tko/migrations/030_add_test_comparison_attributes.py new file +20 -0
@@ -0,0 +1,20 @@ +UP_SQL = """ +CREATE TABLE `test_comparison_attributes` ( + `id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, + `testname` varchar(255) NOT NULL, + `owner` varchar(80) NOT NULL, + `attributes` varchar(1024) NOT NULL, + `testset` varchar(1024) NOT NULL +); +""" + +DOWN_SQL = """ +DROP TABLE IF EXISTS `test_comparison_attributes`; +""" + +def migrate_up(manager): + manager.execute(UP_SQL) + + +def migrate_down(manager): + manager.execute(DOWN_SQL)
Signature · 1 line
1.6.2.5