oFono upstream test results_20130110
Hello all,
First of all, I would like to wish you a happy new year and my best wishes to all of you for 2013!
This has been a long time I have not sent an updated test report about oFono upstream but this last 2 months I was busy on other tasks.
Please find the test report of oFono v1.12 commit 7b97a8a.
During this testing, we ran 462 functional positive cases. 358 cases passed and 100 are blocked due to missing software/missing hardware/Feature not supported. The pass rate is 77% (-1%) which is similar to previous test cycle (v1.11 commit 7b97a8a).
If you have any comment about this report, contact me please.
Test Objective
The aim of this test cycle was to validate the state of oFono upstream by testing all its features over all material we had (according to what feature was supported and by priority order: 3G dongle and phonesim). oFono has been installed and tested on Ubuntu 12.04 device.
Test Environment
oFono: v1.12 (updated to commit 7b97a8a)
usb_modeswitch: v1.2.3
modeswitch data: 20120120
Hardware: Laptop
Ubuntu: v12.04
Modem: Huawei E173u-2 - Operator & SIM Card: several french and UK SIM Cards according to features
Phonesim: v1.19 (updated to commit d7f1f1a)
Issue Summary
New bug: 1
Bug 25952 - oFono crashes when doing a SIM refresh (Reset, Initialization and or File change) with Phonesim
Known bug: 0
None
Closed bug: 0
None
Test Result
SUMMARY
Total Test Case 462
Passed 358
Failed 0
Blocked 100
TCs completed 100,0%
Run rate 78%
Pass rate total 77% (-1%)
Blocked rate total 22%
Pass rate of executed 99% (-1%)
FEATURES TOTAL PASS FAIL BLOCKED
PASS % MODEM USED
Modem 16 16 0 0
100% 3G Dongle
SIM 20 20 0 0
100% 3G Dongle
Phonebook 1 1 0 0
100% 3G Dongle
Network 18 18 0 0
100% 3G Dongle
Radio 22 17 0 5
77% 3G Dongle
Connectivity 28 28 0 0
100% 3G Dongle
Voice Calls 50 40 0 10
80% mainly on 3G Dongle (using appropriate SIM Card)
Messaging 24 22 0 2
92% 3G Dongle (using appropriate SIM Card)
Smart Messaging 15 15 0 0
100% Phonesim
Message Waiting 8 8 0 0
100% Phonesim
Cell Broadcast 15 12 0 3
80% Phonesim
Call Settings 13 9 0 4
69% 3G Dongle
Supplementary Services 59 56 0 3
95% mainly on 3G Dongle (using appropriate SIM Card)
Call Meter 15 5 0 10
33% Phonesim
Call Barring 49 33 0 16
67% 3G Dongle (using appropriate SIM Cards)
Call Forwarding 5 5 0 0
100% Phonesim
SimToolKit 104 53 4 47
51% Phonesim
Please find details in the attached file.
Notes
1% of cases are failed due to:
Bug <https://bugs.meego.com/show_bug.cgi?id=25952> 25952 - I have observed one regression in STK and it is possible the guilty is Phonesim which has not been updated whereas lots of changes have been submitted in oFono due to recent GCF pre-certification work. In order to clarify the origin of this issue, I have first entered the bug for oFono (Please see details in Issue Summary section)
22% of cases are blocked due to:
Missing software (e.g.: Missing feature in Phonesim or missing a visual application for some blocked feature as Setup Idle ModeText, Select Item, Send SMS, etc.)
Missing hardware (e.g.: Missing other SIM (having appropriate service activated) or oFono integrated in an end user device (e.g. smartphone) in order to be able to test certain feature or case with the network simulator (e.g.: PIN/PIN2 cases, Call Barring for specific scenarios etc.).
Feature not supported by oFono
Best regards,
Nicolas