GUI Testing on Smart Devices – Testing Guidelines

As “First impression is the last”, so GUI(Graphical User Interface) does matter and creates a lot of difference. Importance of decent and attractive GUI can be felt more significantly in smart devicesenvironment where screen size is much small.

GUI testing can be toughest part especially while testing on smart device. You should pay full attention to the GUI while testing on smart devices and surely it is an important task that deserves significant time and resource allocation.

Practical Tips for Testing GUI on Smart Devices:

For me, while testing GUI, all the controls are accused. I raise questions why they are there on the screen and I try to answer these questions. I argue in opposition and favor of the controls one by one and I do all this without discussing with someone else. It is the time when I’m wearing multiple hats, Controls are accused and I’m the Prosecutor , I’m the Defense Lawyer and I’m the Judge and during all this process a control must have valid and solid reasons in its favor to be there on screen and consume space. I suggest you to try it and it will help you to decide which controls to display on the screen.

There also come the situations where you are given an already built GUI to test. In such situations also think about the missing controls, the controls that will add value to the screen and compare their importance with the current ones. If you think you need to make a change go ahead.

Once you have decided which controls will be shown on the screen, think thoroughly about size, style and location of the controls on the screen and more important how user will interact with them?

3 important factors to be considered while testing GUI on Smart Devices:

Size:
There are too many variations in screen sizes and available resolutions. In smart devices especially, controls sizes are not static, they have relation to the available screen size.

While testing, make sure that controls size looks esthetically good and control is completely visible on the screen without any sc

rolling. Test the GUI on different devices with different screen sizes and resolutions.

Emulators are good for this purpose but nothing matches the real device. So make sure that you test on at least two or three real devices. Also don’t forget to test on landscape and portrait orientations if the device supports it.

Style:
Definitely your application has a specific design. And style of the controls should match with that design. You might have seen many applications where some controls e.g. panels have round edges and text boxes in them have sharp edges. Although this type of issues don’t affect the usability or functionality but still a consistent look of the application helps to build a friendly relation between the application and the user.

Relatively more important thing in style is font on the different pages. Most of the times, we focus the text that is visible in normal situations and ignore the text that appears in specific situations. Success and Failure messages are an example of such type of text.

Another factor, important in style is relation between the font color and the situation in which text is displayed. For example Red color is used for Error messages, Green for success, Yellow for warnings and Blue (now a day occasionally) for hyperlinks.

Location:
Location and position are the two words that are used alternatively and it is interesting that they are further used to convey two different concepts that are explained below.

1. Sometimes it is the area on the screen where a control appears. For example Header is located on Top of the page, Labels are Left Aligned, and Text boxes are Right Aligned etc. Here text in bold are relative positions of the controls

2. Sometimes it is the order of a control among the other controls. For example while getting personal info, First Name is followed by the last name or format of controls to ask for a US address should be in order ZIP, City, State.

For both these situations, make sure that everything is logical and shows a good aesthetic sense.

Forgot something even more important. There are situations where one or more controls appear on more than one screen, in this situation make sure that they appear on same location and in the same order on all the pages.

Mobile Application Testing

Introduction to Mobile Application Testing:

Gone are the days when the telephone used to be an appliance that sat in a corner and had to ring to get our attention or a computer was a machine only few people used – they are now an extension of our being- a window to the world and virtual servants that do as they are told. Computers were a rage and changed how we humans thought, behaved, learnt and existed.

Types of Mobile Testing

There are broadly 2 kinds of testing that take place on mobile devices:

#1. Hardware testing:

The device including the internal processors, internal hardware, screen sizes, resolution, space or memory, camera, radio, Bluetooth, WIFI etc. This is sometimes referred to as, simple “Mobile Testing”.

#2. Software or Application testing:

The applications that work on mobile devices and their functionality is tested. It is called the “Mobile Application Testing” to differentiate it from the earlier method. Even in the mobile applications, there are few basic differences that are important to understand:

a) Native apps: A native application is created for use on a platform like mobile and tablets.
b) Mobile web apps are server-side apps to access website/s on mobile using different browsers like chrome, Firefox by connecting to a mobile network or wireless network like WIFI.
c) Hybrid apps are combinations of native app and web app. They run on devices or offline and are written using web technologies like HTML5 and CSS.

There are few basic differences that set these apart:

• Native apps have single platform affinity while mobile web apps have cross platform affinity.
• Native apps are written in platforms like SDKs while Mobile web apps are written with web technologies like html, css, asp.net, java, php.
• For a native app, installation is required but for mobile web apps, no installation is required.
• Native app can be updated from play store or app store while mobile web apps are centralized updates.
• Many native app don’t require Internet connection but for mobile web apps it’s a must.
• Native app works faster when compared to mobile web apps.
• Native apps are installed from app stores like Google play store or app store where mobile web are websites and are only accessible through Internet.

The rest of the article is going to be about Mobile Application Testing.

Significance of Mobile Application Testing

Testing applications on mobile devices is more challenging than testing web apps on desktop due to

• Different range of mobile devices with different screen sizes and hardware configurations like hard keypad, virtual keypad (touch screen) and trackball etc.
• Wide varieties of mobile devices like HTC, Samsung, Apple and Nokia.
• Different mobile operating systems like Android, Symbian, Windows, Blackberry and IOS.
• Different versions of operation system like iOS 5.x, iOS 6.x, BB5.x, BB6.x etc.
• Different mobile network operators like GSM and CDMA.
•  Frequent updates – (like android- 4.2, 4.3, 4.4, iOS-5.x, 6.x) – with each update a new testing cycle is recommended to make sure no application functionality is impacted.

As with any application, Mobile application testing is also very important, as clientele is usually in millions for a certain product – and a product with bugs is never appreciated. It often results in monetary losses, legal issue and irreparable brand image damage.

Basic Difference Between Mobile and Desktop Application Testing:

Few obvious aspects that sets mobile app testing apart from the desktop testing

• On desktop, the application is tested on a central processing unit. On a mobile device, the application is tested on handsets like Samsung, Nokia, Apple and HTC.
• Mobile device screen size is smaller than desktop.
• Mobile devices have less memory than desktop.
• Mobiles use network connections like 2G, 3G, 4G or WIFI where desktop use broadband or dial up connections.
• The automation tool used for desktop application testing might not work on mobile applications.

Types of Mobile App Testing:

To address all the above technical aspects, the following types of testing are performed on Mobile applications.

• Usability testing- To make sure that the mobile app is easy to use and provides a satisfactory user experience to the customers
• Compatibility testing- Testing of the application in different mobiles devices, browsers, screen sizes and OS versions according to the requirements.
• Interface testing- Testing of menu options, buttons, bookmarks, history, settings, and navigation flow of the application.
• Services testing- Testing the services of the application online and offline.
• Low level resource testing: Testing of memory usage, auto deletion of temporary files, local database growing issues known as low level resource testing.
• Performance testing- Testing the performance of the application by changing the connection from 2G, 3G to WIFI, sharing the documents, battery consumption, etc.
• Operational testing- Testing of backups and recovery plan if battery goes down, or data loss while upgrading the application from store.
• Installation tests- Validation of the application by installing /uninstalling it on the devices.
• Security Testing- Testing an application to validate if the information system protects data or not.

Mobile Application Testing Strategy

The Test strategy should make sure that all the quality and performance guidelines are met. A few pointers in this area:

1) Selection of the devices - Analyze the market and choose the devices that are widely used. (This decision mostly relies on the clients. The client or the app builders consider the popularity factor of a certain devices as well as the marketing needs for the application to decide what handsets to use for testing.)

2) Emulators – The use of these is extremely useful in the initial stages of development, as they allow quick and efficient checking of the app. Emulator is a system that runs software from one environment to another environment without changing the software itself. It duplicates the features and work on real system.

Types of Mobile Emulators

• Device Emulator- provided by device manufacturers
• Browser Emulator- simulates mobile browser environments.
• Operating systems Emulator- Apple provides emulators for iPhones, Microsoft for Windows phones and Google Android phones

List of few free and easy to use mobile device emulators

i. iPhone Tester – All you need to do with this is – enter the URL in search box and you can see the real time preview of how it appears on an iPhone.

ii. Mobile Phone Emulator – Used to test handsets like iPhone, blackberry, HTC, Samsung etc.

iii. MobiReady – With this, not only can we test the web app, we can also check the code.

iv. Responsivepx – It checks the responses of the web pages, appearances and functionality of the websites.

v. Screenfly – It is a customizable tool and used to test websites under different categories.

3) After a satisfactory level of development is complete for the mobile app, you could move to test on the physical devices for a more real life scenarios based testing.

4) Consider cloud computing based testing: Cloud computing is basically running devices on multiple systems or networks via Internet where applications can be tested, updated and managed. For testing purposes, it creates the web based mobile environment on a simulator to access the mobile app.

Pros:

• Backup and recovery- Cloud computing automatically takes back up of your data from remote location making recovery and restoring of data easy. And also, the storage capacity is unlimited.
• Clouds can be accessed from different devices and anywhere.
• Cloud computing is cost efficient, easy to use, maintain and update.
• Fast and quick deployment.
• Web based interface.
• Can run the same script on several devices in parallel.

Cons

• Less control- Since the application runs on remote or third party environment, user has limited control and access over the functions.
• Internet connectivity issues- the setup is on Internet. Network issues affect the availability and functioning
• Security and privacy Issues- Cloud computing is an Internet computing and nothing on Internet is completing secure, so chances of data hacking are more.

5) Automation vs. Manual testing

• If the application contains new functionality, test it manually.
• If the application requires testing once or twice, do it manually.
• Automate the scripts for regression test cases. If regression tests are repeated, automated testing is perfect for that.
• Automate the scripts for complex scenarios which are time consuming if executed manually.

Two kinds of automation tools are available to test mobile apps:

Object based mobile testing tools- automation by mapping elements on the device screen into objects. This approach is independent of screen size and mainly used for Android devices.

• Eg:- ranorex, jamo solution

Image based mobile testing tools- create automation scripts based on screen coordinates of elements.

• Eg:- Sikuli, Egg Plant, RoutineBot

6) Network configuration is also necessary part of mobile testing. It’s important to validate the application on different networks like 2G, 3G, 4G or WIFI.

Test Cases for Testing a Mobile App

In addition to functionality based test cases, Mobile application testing requires special test cases which should cover following scenarios.

• Battery usage- It’s important to keep a track of battery consumption while running application on the mobile devices.
• Speed of the application- the response time on different devices, with different memory parameters, with different network types etc.
• Data requirements – For installation as well as to verify if the user with limited data plan will able to download it.
• Memory requirement- again, to download, install and run
• Functionality of the application- make sure application is not crashing due to network failure or anything else.

Download Some Sample Test Cases for Testing Mobile Applications:

=> Download Mobile app sample test cases

Typical activities and proceedings in Testing Mobile Application

The scope of the testing depends on the amount of requirements to be checked or the extent of changes made to the app. If the changes are few, a round of sanity testing will do. In case of major and/or complex changes, a full regression is recommended.

An example application testing project: ILL (International Learn Lab) is an application designed to help admin, publisher to create websites in collaboration. Using a web browser, instructors choose from a set of features to create a class that meets their requirements.

Mobile Testing process:

Step #1. Identify the types of testing: As ILL application is applicable for browsers, so it’s mandatory to test this application on all supported browsers using different mobile devices. We need to dousability, functional and compatibility testing on different browsers with the combinations of manual and automation test cases.

Step #2. Manual and Automated testing: The methodology followed for this project is Agile with the iteration of two weeks. Every two weeks dev. team releases a new build to testing team and testing team will run their test cases on QA environment. Automation team creates scripts for set of basic functionality and runs the scripts that help determine if the new build is stable enough to test. The Manual testing team will test the new functionality.

JIRA is used for writing of acceptance criteria; maintaining of test cases and logging /re-verification of defects. Once the iteration gets over, iteration planning meeting held where dev. Team, product owner, business analyst, and QA team discuss what went well andwhat needs to improve.

Step #3. Beta Testing: Once the regression testing is completed by the QA team, the build moves into UAT. User Acceptance Testing is done by the client. They re-verify all the bugs to make sure every bug was fixed and the application is working as expected on every approved browser.

Step #4. Performance test: Performance testing team tests the performance of the web app using JMeter scripts and with different the loads on the application.

Step #5. Browser testing: The web app gets tested across multiple browsers- both using different simulation tools as well as physically using real mobile devices.

Step #6. Launch plan: After every 4th week the testing moves into staging, where a final round of end to end testing on these devices is performed to make sure the product is ready for production. And then, it goes Live!

Conclusion

Designing the right test strategy, choosing the right mobile simulators, devices and mobile testing tools can make sure that we have 100% test coverage and help us include security, usability, performance, functionality and compatibility based tests into our test suites.

Well, this has been our effort to fulfill multiple requests from our readers on a mobile application testing guide.

Robotium - Android Application UI Testing Tool

Robotium is the android test automation framework for testing native and hybrid android applications. It provides simple API to write UI automation scripts. However, Number of tools are available for testing android applications, Robotium is the most commonly used Android testing tool.

Robotium can be used to write functional, system and user acceptance testcases. Robotium Testcases can be executed in Android emulator as well as the Android real device.

What is covered in this Robotium Tutorial:

• Benefits
• Prerequisites
• Creating Robotium Project
• Creating Robotium Class
• Some Robotium Methods
• Locating  Elements in Android application
• Sample Code
• Executing Robotium Project

Benefits of Robotium

• Easy to write
• Simple API (All Methods are available only in Solo Class)
• Automatic Delays & Timings
• No need to write code , when navigating from one activity to another activity.
• Test Android native apps as well as hybrid app.
• Able to handle multiple android activities.
• Less time to write tests as the API is simple.
• Testcases are Robust , due to runtime binding to UI components
• Fast testcase execution.
• Integrates easily with Maven and ANT.

Prerequisites

1) Download and Install JAVA
– Download Java from this page.
– Add java libraries to the PATH and set JAVA_HOME to the root of java installation directory in your environment.

2) Download ADT Bundle

• Download ADT bundle from this page.
• Extract ADT bundle zip and put it in a folder.
• Set ANDROID_HOME to the root of ADT bundle folder in your Environment.

Creating a Robotium Project

Only few steps required to create a project,

Step #1: Open eclipse contains your android application to be tested.

(Click on image to enlarge)

Step #2: Right click on the android project -> Android Tools and click on new Test project.

Step #3: Give name for the Test project and click Next button.

Step #4: Select The Application under test as the target and click finish button.

Step #5: Test project will be created in the eclipse workspace.

Step #6: Download Robotium solo jar from here.

Step #7: Right click on the Test project in the workspace Go to Build path and click Configure build path.

Step #8: Switch to libraries tab, click on “Add external jars” option and browse the downloaded Robotium jar file and add it to the libraries and click “OK”.

Step #9: Robotium Test project is created successfully. Now we can create classes under the project and start writing the test cases.

Creating Robotium Class

#1: Right click on the package under src directory in the Test project, and create a new class.

#2: Import the Main Activity class the Robotium test project.
Syntax:

1	import com.sasi.attendanceproject.Home;

#3: New class will inherit properties from ActivityInstrumentationTestCase2 class

1 2	public class AttendanceTest extends ActivityInstrumentationTestCase2 <home>

(Note: Here, Home is the activity to be tested in the android application)

#4: Create Instance for Solo class as below

1	private Solo solo;

#5: Create a Constructor for the Test class, as below

1 2 3

public AttendanceTest() {     super(Home.class);     // TODO Auto-generated constructor stub

#6: Create setUp and tearDown methods , as below
Setup method is used to Initiate the Instrumentation

1 2 3 4

public void setUp()throws Exception {    solo=new Solo(getInstrumentation(), getActivity()); }

Teardown method is used to close the activity after the test has been completed.

1 2 3

public void tearDown() throws Exception { solo.finishOpenedActivities(); }

Some Robotium Methods

#1. assertCurrentActivity (text,Activity)
This method verifies whether the current activity is the activity which is passed as the send parameter.
Syntax

1	solo.assertCurrentActivity("Current Activity", Home.class);

#2. clickOnButton(text)
This method will click on the button with the specified Text.
Syntax:

1	solo.clickOnButton("ADMIN");

#3. clickOnButton(int)
This method will click on the button with the specified index.
Syntax:

1	solo.clickOnButton(2);

#4. waitForText(text)
This method will wait until the text appearing on the activity.
Syntax:

1	solo.waitForText("Creating New Password");

#5. enterText(int, text)
This method will type the text passed as the second parameter to the specified index editbox.
Syntax:

1	solo.enterText(0,"test");

#6. clickOnCheckbox(int)
This Method will click on the checkbox with given index.
Syntax:

1	solo.clickOnCheckBox(0);

#7. clickOnRadioButton(int)
This Method will click on the Radio button with the given index.
Syntax:

1	solo.clickOnRadioButton(1);

#8. clickOnImage(int)
This Method will click on the image with the given index.
Syntax:

1	solo.clickOnImage(1);

#9. clearEditText(int)
This Method will clear the text in the edit box with the given index.
Syntax:

1	solo.clearEditText(0);

#10. waitForText(text)
This Method will wait until the given text is appearing on the activity.
Syntax:

1	solo.waitForText(“Robotium”);

Example Program

Locating Elements in Android Application

Step – 1
Open Eclipse containing Android Application to be Tested.

Step – 2
Expand the project , Go to res ? Layout folder and double click on the activity XML file, you want to automate. It will open the designed activity on the eclipse editor.

Step – 3
Locating Button Element

Locating Element By ID
Click on the Element , you want to locate. On the right side properties panel, you can find the ID of that element. (ignore @+id/)

(Click on image to enlarge)

In the Above figure ID of the element is, btnadmin.

Corresponding Robotium code to locate this button is,

1	solo.clickOnButton(R.id.btnadmin);

Locating Element By Position

In case, if ID is not available then you can locate the element using the order. If the button is in the second position, Then you can locate the button by,

1	solo.clickOnButton(1);

Locating Element By Text

Even you can click on the button using the text displayed on the button.

1	Solo.clickOnButton(“ADMIN”);

Locating Text box Element 
Using Similar way click on the text box you want to enter data and find the position of the text box from the right side.

1 2	Solo.enterText(0,”Anitha”); //Username textbox is in 1st position

1 2	Solo.enterText(1,”test”);   //Password textbox is in 2nd position

Locating a Radio Button Element
Click on the Radio Button Element , you want to click and find out the position of the radio button.

1 2	Solo.clickOnRadioButton(1); //It will location radio button in the second position.

Sample Code

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

package com.sasi.attendanceproject.test; import android.test.ActivityInstrumentationTestCase2; import android.widget.EditText;   import com.robotium.solo.By; import com.robotium.solo.Solo; import com.robotium.solo.WebElement; import com.sasi.attendanceproject.Home;   public class AttendanceTest extends ActivityInstrumentationTestCase2<Home>{     private Solo solo;     public AttendanceTest() {         super("com.sasi.attendanceproject.Home",Home.class);         // TODO Auto-generated constructor stub     }       public void setUp()throws Exception{         solo=new Solo(getInstrumentation(),getActivity());     }     public void testAttendance()throws Exception{         //solo.assertCurrentActivity("Current Activity", Home.class);         solo.waitForWebElement(By.id("btnadmin"));         solo.clickOnButton("ADMIN");         solo.clickOnButton(0);         solo.waitForText("Creating New Password");         solo.enterText(0, "test");         solo.enterText(1, "test");         solo.clickOnButton("Okay");         solo.waitForText("Attendance Login");         solo.enterText(0, "Anitha");         solo.enterText(1, "test");         solo.clickOnButton("Login");         solo.waitForWebElement(By.id("btnaddnew"));         solo.clickOnButton("Add New Details");         solo.waitForText("Enter the Employee Details");         solo.enterText(0, "Anitha");         solo.enterText(1, "6");         solo.enterText(2, "Testing Engineer");         solo.clickOnRadioButton(1);         solo.clickOnButton("Okay");         solo.waitForWebElement(By.id("tvempID"));         System.out.println(solo.getText(0));       }     public void tearDown()throws Exception{         solo.finishOpenedActivities();}}

Executing Robotium Project

From Eclipse:

Right click on the project and select Run As -> Android Junit Test

From Command Prompt:

Step 1:
Cd to your Android Test Project Directory

Step 2:
Run the following command,

1 2	adb shell am instrument -w <<package name of your test class>>/android.test.InstrumentationTestRunner

Conclusion

• Robotium is the most commonly used Android Test automation tool.
• Robotium Testcases can be executed on Android Emulator as well as the Real device, we don’t need to write any specific configuration code to run Robotium test cases on Real device.
• Robotium Can be easily written in maven project also, and it can be run through continuous integration tools.
• Thus, Robotium is very useful in writing easy/simple Android Test automation scripts.