1.

Field Testing

Field-testing is one of the important phases of mobile handset testing. Until now whatever the testing was discussed, they were all carried out in the laboratory condition. However the actual usage of the mobile handset is outside the laboratory such as office crowded public places, remote places, bus station, railway station, airport, harbor, on the Sea (?) The requirement is that the mobile handset works as per the product requirements in all the places and whether conditions. 1.1.1 Objective The mobile handset behaves as per the product requirements, in all the places and in all the environmental conditions. 1.1.2 Pre-requisites o o o The mobile phone with loaded software released for field-testing. Commutation means, such as car and the driver capable of moving at the required speed. Various SIM cards with required level of access.

1.1.3 Planning During the planning phase, various activities related to the Field-testing are planned. In other words, tests plan document, test preparation document, Test coverage matrix and Test reporting etc is planned. The activity involves, a series of discussions covering: Identification of geographical locations where the field-testing needs to be performed. Detailed Time plan/Schedule for executing the field-testing as it depends on field conditions. Test cases availability/identification (within the organization or procurement from third party) or fresh development of test cases Test environment availability, hardware and software requirements. Mobility requirements. Test reporting simple excel reports or reporting through tools like Rational Test manager Effort and mandatory skills requirement. This includes the forming of the team, effort estimation for the various testing phases.

1.1.4 Test design and Implementation The test design and implementation of Field test cases is actually identifying the different user-case scenario testing. These test cases are documented in an excel sheet and are executed during the execution phase. 1.1.5 Test execution The test execution process in the field-testing is almost same as that of system testing except that the test cases are executed in the field. The various test cases include, making GSM/GPRS calls when going in a car with say 90miles per hour, sending messages, taking picture and sending the picture by MMS, web browsing etc. the tests are actually user-case scenarios. An example of Field-testing is illustrated in the Appendix 1.1.6 Dos and Donts This section describes some of the best practices in the area of field-testing. Dos: During the planning phase o Prepare a Detailed time plan with certain amount of buffer time for test reporting. This will help if there occurs any expected delay in testing. o Plan a mechanism or process with which all the unclear requirements are made clear during the course of Test design and implementations. I.e. before starting the testing. During the Test design phase o Design all the possible test cases for all the used case scenarios. o In case if there are any product specific requirements, do consider them in designing the test cases. o Associate all the test cases to the requirement being tested. Maintain a document/spreadsheet about this cross-reference. o Look for the scope of Test automation in the design phase. During The testing Phase o Use the different SIM cards from different network service providers and try to execute the tests on all the available networks. This will help in analysing the source of defect i.e. whether it is because of handset or because of network. o Log the defect as soon as it is detected and notify the concerned developer. o Capture all the warning/observations and use them during the testing of later versions of the software. o In the case of user scenario test case, check the behavior on other competitor phones. This gives a direction for improvement. Donts: During the planning phase o Do not assume in case of ambiguity in requirements This will lead to unexpected and unwanted consequences. o Do not assume 100% quality on the third party games and applications downloaded from the service provider as a part of testing.

During the Test design phase o Do not make design discussions open-ended (i.e. without any conclusions) ensure that every design discussion meetings ends with certain conclusions start the next discussions with the previous conclusions. o Do not use prototype as product. During The testing Phase o Before logging any defect, do verify that the test case is valid and the defect is valid one. o Do not log duplicate defects.

Field Testing Document 1] CS / PS / Combined / RAU(Radio Access Unit) / Periodic / LAU Combined Registration Procedure (CS+PS) (With Power On Attach) Registration Procedure (CS+PS) (Without Power On Attach) Periodic Location Update / Routing Area Update / Periodic Routing Area Update MO / MT Voice Call MO / MT Voice Call (AMR 12.2kbps) MO / MT Voice Call (Signaling over FACH) GSM / GPRS Call Scenarios MO Packet data Call (FTP, PC Browsing, Ping, E-Mail Download, Streaming and MMS) Multiple Simultaneous PS Calls (multiple primary PDP context activation Circuit Switch Data (CSD) 2] Video Telephony Basic Test Cases 1. MO / MT Video Telephony Call and testing of features like show me, re-try to speech etc. 2. Successful MO VT call followed by the normal from Origination side. Verify Audio and video. 3. Called party rejects Call, call should be terminated, with "reject" cause on MO. At the MT side also a successful release should happen. 4. Called party does not answer; Call should be terminated with "no answer" cause at MO. At the MT side also a successful release should happen. 5. Called party accepts call, but does not press show. Called party's video should not be visible at MO. 6. Called party busy in another call. Call should be terminated with cause "user busy". 7. Called party disconnects after accepting, before "show me" button appears.Call should be terminated successfully at MO .

8. Collect MO connection time (csd, h324m, csd + h324m),verify that signalling time is less than 6 secs after called party presses "accept" key. 9. Calling party disconnects the call before Accept/ Reject button appears on MT handset. Call should be terminated successfully. 10. Calling party Disconnects, before the called party presses accept / Reject button. 11. VT Call is not successful, Called party (terminating) mobile is not a 3G324M UE. 12. VT Call not success, because, Called party already in Voice Call. 13. VT Call not success, because, Called party already in Video Call. 14. Mute and Unmute audio at MT. 15. Mute and Unmute video at MT .Check that video gets properly manipulated at the MO. (Note down the time for the video to recover after unmute is pressed). 16. Mute and Unmute (Audio and Video) at MT.Verify that audio and video is properly manipulated at the MO. 17. Use the bar on the left of handset to increase / decrease the volume. 18. Check video quality at MO/MT. If video quality is commercial quality then success, else failure (These tests have to be done in High Load, Avg Load and Low load conditions). 19. Log End to end delay for audio and video. Audio delay should be less than 2 sec. 20. Video Call active, Insert Hand-Free Kit, Remove it. Repeat it a number of times. VT and PS interaction test cases [VT and WAP] 1. MO VT calls while WAP connection is active: Initiate a WAP connection; once the connection is active now make a MO VT Call from test mobile. 2. MT VT calls while WAP connection is active: Initiate a WAP connection, once the connection is active now make a VT call to the test mobile from other mobile. 3. Reject MT VT call when WAP connection is active: Intiate WAP connection on the test mobile, Make a VT call from the other mobile to the Test mobile and Reject the incoming VT call from the test mobile. 4. Disconnect WAP connection after MO VT call is active:

Initiate a WAP connection from the test mobile, once the connection is active, make a MO VT call from test mobile to other mobile, Once the call is active disconnect the PDP (WAP) connection from MO. 5. Disconnect WAP connection after MT VT call is active: Initiate a WAP connection from the test mobile, once the connection is active, make a VT call from other mobile to the test mobile. Once the call is active disconnect the PDP (WAP) connection from MO. 6. Disconnect MO VT call while WAP connection is active: Initiate a WAP connection from test mobile, once the connection is active, make a MO VT call from test mobile to the other mobile . Once the call is active disconnect the VT call from MO side.

DTMF test cases Press the Number Buttons, during video call and check that video call is not affected. Verify the DTMF from Traces.

SMS - MO/MT SMS In Idle Mode (> and <= 160 Characters) - MT SMS while MO Voice Call is Active - MT SMS while MT Voice Call is Active - MT SMS while PS Data Call is Active - MO SMS while MO Voice Call is Active - MO SMS while MT Voice Call is Active - MO SMS while PS Data Call is Active - Check for Delivery notification - Successful delivery and receipt of a Multi Page (7 to 8 pages) SMS in Normal Mode - Successful delivery and receipt of a Multi Page (7 to 8 pages) SMS in Dedicated mode

MMS MO/MT MMS (Idle Mode) MT MMS while MO Voice Call is Active MT MMS while MT Voice Call is Active MT MMS while PS Data Call is Active MO MMS while MO Voice Call is Active MO MMS while MT Voice Call is Active MO MMS while PS Data Call is Active

Manual Network Mode Change Change Network Mode from Automatic to UMTS, GSM900/1800 and GSM1900 Change Network Mode from UMTS to Automatic, GSM900/1800 and GSM1900 Change Network Mode from GSM900/1800 to Automatic, UMTS, and GSM1900 Change Network Mode from GSM1900 to Automatic, UMTS, and GSM900/1800 Keep switching between 3G and 2G and automatic modes at a rapid rate (UE be not allowed to register with either of the RATs)

PLMN Selection Tests 1. Network Selection Failure when camping on to a PLMN which is not allowed. 2. Repeated forced termination of Manual PLMN selection by pressing the end key. 3. Repeated cancellation of Manual PLMN selection by pressing the back key. Low Signal Area Tests 1. Make MO calls to the landline. Record the pass % in the results field. Give subjective feedback. 2. Make MT calls. Record the pass %.Record the pass % in the results field. Give subjective feedback.

3. PC browsing in a low signal area.(See if FTP upload and Download work). Give subjective feedback. 4. WAP browsing in a low signal area. Give subjective feedback. 5. Streaming in a low signal area. Give subjective feedback. Do Manual PLMN Selection. Initiate and terminate - 15 times. Don't allow the UE to register to any network. After this is done check that UE doesn't enter Continuous Searching and should also camp on to a PLMN if allowed to complete the search. Check if Cell selection /re-selection in idle mode. Recovery from No Service state: Let the UE be in No Service state for extended period of time. After this is done bring the UE back to a good signal area. The UE should recover properly from the No Service state.

5. While retrieving the MMS in 3G, receive Voice Call. 6. Make Multiple Voice Calls(6 Calls-> 5 on HOLD and 1 Active and vice versa) . User Initiated PS Call in GPRS via IrDA, receives Voice Call. FTP download throughput test Case (a): Power on the UE. Then connect the USB cable to the UE and the PC. Make 10 FTP downloads (10 PS calls). Case (b): Make 10 ftp downloads powering off and powering on the UE after the completion of every download.