oFono upstream test results_20130517
Hello all,
Please find the test report of oFono v1.12 commit 1c75edf.
Due to Network Simulator License issue, no report has been sent last month, I'm sorry for the inconvenience. This problem is now resolved.
During this testing, we ran 462 functional positive cases. 372 cases passed, 1 failed and 89 are blocked due to missing software/missing hardware/Feature not supported. The pass rate is 81% (=) which is equal than previous test cycle (v1.12 commit 4fcd084). No new issue has been found.
If you have any comment about this report, contact me please.
Test Objective
The aim of this test cycle was to validate the state of oFono upstream by testing all its features over all material we had (according to what feature was supported and by priority order: Smartphone with real SIM or with Network Simulator, on Laptop with 3G dongle or with Phonesim). oFono has been installed and tested on smartphone device running Tizen 2.0 and Ubuntu 12.04 device.
Test Environment
For all Setup:
oFono: v1.12 (updated to commit 1c75edf)
- Smartphone device Setup:
Image: Tizen Mobile 2.0
Operator & SIM Card: several french and UK SIM Cards according to features
- Laptop + 3G Dongle/Phonesim:
usb_modeswitch: v1.2.3
modeswitch data: 20120120
Hardware: Laptop
Ubuntu: v12.04
Modem: Huawei E173u-2 - Operator & SIM Card: several french and UK SIM Cards according to features
Phonesim: v1.19 (updated to commit c94e6c0)
- Laptop + HFP
HFP Server:
Hardware: Laptop
Ubuntu: v12.04
Bluez 4.101 (updated to commit 67ef3ac)
oFono has been installed with "--enable-bluez4" option instead of using default bluez5 plugin (unable to make HFP using this one)
HFP Client: Android JB device
Issue Summary
New bug: 0
None
Known bug: 1
Bug 25982 <https://bugs.meego.com/show_bug.cgi?id=25982> - Going back from a select item list releases the session
Closed bug: 0
None
Test Result
SUMMARY
Total Test Case 462
Passed 372
Failed 1
Blocked 89
TCs completed 100,0%
Run rate 81%
Pass rate total 81% (=)
Blocked rate total 19% (=)
Pass rate of executed 100%
FEATURES TOTAL PASS FAIL
BLOCKED PASS % MODEM USED
Modem 16 16 0
0 100% Smartphone using real SIM and Laptop +
HFP Modem
SIM 20 20 0
0 100% Smartphone using real SIM
Phonebook 1 1 0
0 100% Smartphone using real SIM
Network 18 18 0
0 100% Smartphone using real SIM
Radio 22 17 0
5 77% Smartphone using real SIM
Connectivity 28 28 0
0 100% Smartphone using real SIM and Laptop + 3G
Dongle (for PPP tests)
Voice Calls 50 45 0
5 90% Smartphone using real SIM or Network
Simulator
Messaging 24 22 0
2 92% Smartphone using real SIM or Phonesim
Smart Messaging 15 15 0 0
100% Laptop + Phonesim
Message Waiting 8 8 0
0 100% Laptop + Phonesim
Cell Broadcast 15 12 0
3 80% Laptop + Phonesim
Call Settings 13 9 0
4 69% Smartphone using real SIM
Supplementary Services 59 56 0 3
95% Smartphone (using appropriate SIM Card)
Call Meter 15 13 0
2 87% Smartphone using Network Simulator
Call Barring 49 33 0
16 67% Smartphone (using appropriate SIM Cards)
Call Forwarding 5 5 0
0 100% Smartphone (using appropriate SIM Cards)
SimToolKit 104 54 1
49 52% Laptop + Phonesim
Please find details in the attached file.
Notes
Please note that I have found a crash of oFono when running two STK TCs in a row:
Bug <https://bugs.meego.com/show_bug.cgi?id=25998> 25998 - oFono crashes when doing a NAA Initialization+File Change Notification after having done a NAA Initialization+Full File Change Notification with Phonesim.
19% of cases are blocked due to:
- Missing software (e.g.: Missing feature in Phonesim or missing a
visual application for some blocked feature as Setup Idle ModeText, Select Item, Send SMS, etc.)
- Missing hardware (e.g.: Missing other SIM (having appropriate service
activated).
- Feature not supported by oFono
Best regards,
Nicolas