Mobile Application Testing Basics

What is Mobile Application

A mobile application is a software that runs on a mobile device such as a cell phone or MP3 player that will allow the device to perform specific tasks that are typically restricted to PCs.

It also known as downloadables, mobile application are common on most phones, including

inexpensive, entry level models.

Types of Mobile Application

Mobile Applications are mainly divided into three types :

• Browser Based application
• Pre-installed applications
• Installed applications

Mobile applications categories

Communications:

• Email Clients
• IM Clients
• Mobile Web and Internet Browsers
• News/Information Clients
• On­device Portals(Java Portals)
• Social Network Clients

Games:

• Puzzle/Strategy (e.g., Tetris, Sudoku, Mah­jong, Chess, Board Games)
• Cards/Casino (e.g., Solitaire, Blackjack, Roulette, Poker)
• Action/Adventure (e.g., Doom, Pirates of the Caribbean, Role­Playing Games)
• Sports (e.g., Football, Soccer, Tennis, Basketball, Racing, Boxing, Skiing)
• Leisure Sports (e.g., Bowling, Pool, Darts, Fishing, Air Hockey)

Multimedia:

• Graphics/Image Viewers
• Presentation Viewers
• Video Players
• Audio Players
• Streaming Players (Audio/Video)

Productivity:

• Calendars
• Calculators
• Diary
• Notepad/Memo/Word Processors
• Spreadsheets
• Directory Services (e.g., yellow pages)
• Banking/Finance

Travel:

• City Guides
• Currency Converters
• Translators
• GPS/Maps
• Itineraries/Schedules
• Weather

Utilities:

• Profile Manager
• Idle Screen/Screen Savers
• Address Book
• Task Manager
• Call Manager

Challenges in Mobile Application testing

Mobile app marketing is growing now very fast, but in testing field number of challenges arrives due to variation in handsets, phone carriers, networks supported and application written in different languages.

Browser Based Applications: Characteristics of the browser based applications

• Application builds for only mobile application.
• Can be accessed by entering the specific URL in mobile browser.
• No need to installation and UN-installation.
• No involvement of upgrade.

Critical areas for Browser based application:

• Browser based application always expect the connectivity.
• Speed and coverage are the critical aspect.
• Cache related issues.

Pre Installed Application: Characteristics of the pre installed applications

• Application which are shipped as in built software with device
• No download involved
• No installation and un installation involved
• Automatic upgrade can be done

Critical areas for Pre Installed application:

• Prototype testing is very critical for such applications
• Core database affecting directly due to such application crashes
• They can not be uninstall or deleted ever
• Crashes can cause several damage to ROM

Installable Application: Characteristics of the installable applications are

• Can be upgraded
• Application executable file can be received by the wireless media and wired media

Critical areas for Installable application:

• Can be installed and uninstalled in the device
• They can be downloaded from

1. App Stores
2. OAT (over the air)
3. transferred via USB

• Can be transferred via wireless media like Bluetooth, infra etc

Mobile Testing generics:

Testing in different network speed.

1. Low
2. Medium
3. High

Testing during change of network speed

1. Low to high
2. High to low

Testing in different network types

1. 2G (GPRS, CDMA, EDGE)
2. 3G
3. Wi-Fi
4. Different types of plan based on the service provider.

Testing in different battery strategy

1. Critical
2. Low
3. During charging
4. High

Monitoring the battery consumption pattern

1. Observe the battery consumptions where application is running in the background.
2. Observe the battery consumptions where application is running in the foreground.
3. Run the Application for long time.

Monitoring on the memory consumption patterns

1. observes the memory use during launching the applications
2. While application running the app in background
3. While application running the app in foreground
4. During exit the applications.
5. Run the application for long time

Testing on Smart Phone

Here is the list of testing types; user can do on smart phone to validate their applications.

1. Installation testing: Here need focus on the application installation and uninstallation behavior on the device from market place or other resources like through air or download from website.

2. Requirement Functionality Testing: Test the application functionality, behavior of the application according to its design test cases, against its requirement.

3. Widget testing: Test the widget by adding widget, using widget as per functionality i.e. music widget should play music, deleting widget.

4. Phone Interrupt testing: Test the application behavior on receiving/dialing the phone call, SMS & other notifications.

5. Stress testing: Test the application’s performance to generate the different events simultaneously of different frequencies (with the help of programming).

6. UI testing: All mobile platforms have certain submission guidelines to follow before the application can be available commercially. Apple AppStore is very strict on guidelines. Most of the applications submitted are rejected due to small errors on AppStore. We need to verify (UI test) that application is meeting guidelines:

            Guideline for Android: Click  here to see details.

            Guideline for iPhone: click here to see details.

            Guideline for different screen size: click here to see details.

7. Compatibility Test: Developers do the unit testing on the emulators. Testers need to test on actual environment i.e. devices. There are multiple devices of different resolutions & having various OS. We need to check for compatibility on maximum devices.

8. iPhone Testing: Applications for the iPhone are written in Objective-C. Objective-C is an object-oriented programming language which is very similar the C programming language.

Following are some basic scenarios which should be considered while testing mobile application:

• UI of the app is as per the screen size of the device, no text/control should be cutting off.
• Text should be readable.
• Suspend/Resume (Call/SMS/Alarm) when app is running.
• Behavior of app, while mother functionality of device come to existence in between.
• Behavior of app on Flip/Slider closes.
• Behavior of app when memory of the device is almost full (very critical).
• Behavior of when network is not available.
• Backup and restore
• Behavior of app if app is running for longer period of time
• Behavior of app when keys are pressed randomly.
• If app support app/port directed SMS then behavior on receiving such SMS.

Testing Mobile Business Applications: 

Approch	Type of Testing	Manual Testing		Automated Testing on Device
		Using Device	Using Emulators
Stanrd testing	Unit Testing	No	Yes	No
	Integration Testing	No	Yes	No
	System Testing	Yes	No	No
	Regression testing	Yes	No	Yes
Special type of testing to address spacific challenges	Compatibility Testing	Yes	No	Yes
	GUI Testing	Yes	No	No
Type of testing more relavent for enter prise mobile business application	Performence Testing	Yes	No	Yes
	Security Testing	Yes	No	Yes
	Syncronization Testing	Yes	No	No

Testing Checklist for Mobile Applications

Here is a basic check list for mobile app, which help of this we can generate an effective checklist for a application

No.	Module	Sub-Module	Test Case Description	Expected Result
1	Installation		Verify that application can be Installed Successfully.	Application should be able to install successfully.
2	Uninstallation		Verify that application can be uninstalled successfully.	User should be able to uninstall the application successfully.
3	Network Test Cases		Verify the behavior of application when there is Network problem and user is performing operations for data call.	User should get proper error message like “Network error. Please try after some time”
4			Verify that user is able to establish data call when Network is back in action.	User should be able to establish data call when Network is back in action.
5	Voice Call Handling	Call Accept	Verify that user can accept Voice call at the time when application is running and can resume back in application from the same point.	User should be able to accept Voice call at the time when application is running and can resume back in application from the same point.
6		Call Rejection	Verify that user can reject the Voice call at the time when application is running and can resume back in application from the same point.	User should be able to reject the Voice call at the time when application is running and can resume back in application from the same point.
7		Call Establish	Verify that user can establish a Voice call in case when application data call is running in background.	User should be able to establish a Voice call in case when application data call is running in background.
8	SMS Handling		Verify that user can get SMS alert when application is running.	User should be able to get SMS alert when application is running.
9			Verify that user can resume back from the same point after reading the SMS.	User should be able to resume back from the same point after reading the SMS.
10	Unmapped keys		Verify that unmapped keys are not working on any screen of application.	Unmapped keys should not work on any screen of application.
11	Application Logo		Verify that application logo with Application Name is present in application manager and user can select it.	Application logo with Application name should be present in application manager and user can select it.
12	Splash		Verify that when user selects application logo in application manager splash is displayed.	When user selects application logo in application manager splash should be displayed.
13			Note that Splash do not remain for fore than 3 seconds.	Splash should not remain for fore than 3 seconds.
14	Low Memory		Verify that application displays proper error message when device memory is low and exits gracefully from the situation.	Application should display proper error message when device memory is low and exits gracefully from the situation.
15	Clear Key		Verify that clear key should navigate the user to previous screen.	Clear key should navigate the user to previous screen.
16	End Key		Verify that End Key should navigate the user to native OEM screen.	End Key should navigate the user to native OEM screen.
17	Visual Feedback		Verify that there is visual feedback when response to any action takes more than 3 seconds.	There should be visual feedback given when response time for any action is more than 3 second.
18	Continual Keypad Entry		Verify that continual key pad entry do not cause any problem.	Continual key pad entry should not cause any problem in application.
19	Exit Application		Verify that user is able to exit from application with every form of exit modes like Flap,Slider,End Key or Exit option in application and from any point.	User should be able to exit with every form of exit modes like Flap,Slider,End Key or Exit option in application and from any point.
20	Charger Effect		Verify that when application is running then inserting and removing charger do not cause any problem and proper message is displayed when charger is inserted in device.	When application is running then inserting and removing charger should not cause any problem and proper message should be displayed when charger is inserted in device.
21	Low Battery		Verify that when application is running and battery is low then proper message is displayed to the user.	When application is running and battery is low then proper message is displayed to the user telling user that battery is low.
22	Removal of Battery		Verify that removal of battery at the time of application data call is going on do not cause interruption and data call is completed after battery is inserted back in the device.	Removal of battery at the time of application data call is going on should not cause interruption and data call should be completed after battery is inserted back in the device.
23	Battery Consumption		Verify that application does not consume battery excessively.	The application should not consume battery excessively.
24	Application Start/ Restart		1. Find the application icon and select it 2. “Press a button” on the device to launch the app. 3.Observe the application launch In the timeline defined	Application must not take more than 25s to start.
25	Application Side Effects		Make sure that your application is not causing other applications of device to hamper.	Installed application should not cause other applications of device to hamper.
26	External incoming communication – infrared		Application should gracefully handle the condition when incoming communication is made via Infra Red [Send a file using Infrared (if applicable) to the device application presents the user]	When the incoming communication enters the device the application must at least respect one of the following: a) Go into pause state, after the user exits the communication, the application presents the user with a continue option or is continued automatically from the point it was suspended at b) Give a visual or audible notification The application must not crash or hung.

Testing on emulator Vs actual devices

• Mobile software is written on an emulator and eventually runs on the actual device.
• There are less or more subtle differences between emulators and devices (and in particular across devices from different vendors) in terms of supported APIs, hardware and Java Virtual Machine implementation details.
• Uploading and testing the application on each and every device is a tedious, time consuming process.

Emulator Vs Simulator

• You write new program that draws the calculator's display and keys, and when the user clicks on the keys, your programs does what the old calculator did. This is a Simulator
• You get a dump of the calculator's firmware, then write a program that loads the firmware and interprets it the same way the microprocessor in the calculator did. This is an Emulator
  
  The Simulator tries to duplicate the behavior of the device.
  The Emulator tries to duplicate the inner workings of the device.