Android Automation Tools

In Order to Make Automated Tests Effective and Profitable One Should:

• Research the software product under test and the project;
• Clearly determine what tests are to be automated;
• Formulate requirements to the automated tests and the tools for automation;
• Study at least several available and more or less suitable tools for automation;
• Select the most suitable one or more tools basing on the researches;
• Discuss the chosen automation tools with other project parties, explain the choice, and get their approval;
• Proceed to tests automation.

Such approach and sequence of steps would be approved by most of quality assurance experts.

Many different kinds of computers have appeared recently, their appearance started rapid development of software products. The most amazing is evolution of mobile devices; they significantly differ from common personal computers by characteristics, ways and conditions of interaction with them.

Consequently, smartphones and tablets require special mobile applications that differ from desktop ones.

Operating system of most of personal computers is Windows. Among popular mobile operating systems are Android, Apple iOS, Blackberry OS, Windows Phone, Symbian and other.

Top 5 Android Testing Tools:

Let’s explore modern tools for automated testing of applications supported on one of the most widespread mobile operating system – Android.

#1. Robotium Android Testing Tool

(Click to enlarge image)

Robotium is one the first and frequently utilized automated testing tools for software supported on Android.

Robotium is a free Android UI testing tool. It is suitable for tests automation for different Android versions and sub-versions. Software developers often describe it as Selenium for Android. Tests created by Robotium are written in Java. In fact, Robotium is a library for unit tests.

But it takes much time and efforts to create tests by means of Robotium, as one must work with the program source code in order to automate tests. The tool is also unsuitable for interaction with system software; it cannot lock and unlock a smartphone or a tablet. There is no Record and Play function in Robotium, and it does not provide screenshots.

#2. MonkeyRunner Android App Testing

MonkeyRunner is one of popular Android Testing tools used for automating of functional tests for Android software.

This tool is more low-level than Robotium is. One does not have to deal with the source code in order to automate tests. The tests are written in Python, one may use a recording tool for creating tests.

MonkeyRunner can run tests on real devices connected to a PC or emulators. The tool has an API what allows it to control a smartphone, a tablet or an emulator from outside of Android code.

Significant disadvantage of the mobile app testing tool is that it is necessary to write scripts for each device. Another problem of MonkeyRunner is that the tests require adjustments each time when user interface of the tested program is changed.

#3. Ranorex Android Application Testing Tool

(Click to enlarge image)

Ranorex is a good tool for tests automation not only for the latest, but also for early versions and sub-versions of Android, beginning from Android 2.2.

One of Ranorex advantages is its detailed reports with screenshots. It can connect a smartphone or a tablet to Internet via WiFi.

An automated test engineer can elaborate data-driven tests, excluding XML data format, by means of this Android tool. Ranorex Studio enables an automated test engineer to create tests easily, just clicking the mouse. It allows elaborating additional program modules. The modules may be utilized during late development cycles for more complex test scenarios.

It is a commercial mobile application tool; its license price is 1990 EUR. Ranorex searches elements rather slowly; it takes up to 30 seconds to perform such an operation. One must instrument APK files for Ranorex. Otherwise it is impossible to automate tests by means of this tool, as it works only with instrumented APK files.

#4. Appium Android Automation Framework

(Click to enlarge image)

It’s a framework for creation automated tests for iOS and Android. It is a free tool. It supports Android versions from 2.3 and later. Appium utilizes WebDriver interface for tests running. It supports many programming languages, such as Java, C#, Ruby and other which are in the WebDriver library.

It can control Safari and Chrome on mobile devices. This allows to test mobile web sites using Appium and these browsers.

But some automated test engineers complain that it provides poor, insufficient reports. Also its weakness is reduced XPath support on mobile devices.

#5. UI Automator for Android Test Automation

(Click to enlarge image)

This tool has been recently elaborated by Google. It supports Android versions beginning from 4.1. One should select another Android app testing tool in order to automate tests for earlier versions. UI Automator is able to interact with all kinds of software products for Android, including system applications. This enables UI Automator to lock and unlock a smartphone or a tablet.

Scripts created by means of this tool can be executed on many different Android platforms. It allows reproducing complex sequences of user actions.

UI Automator also can utilize external buttons of a device, such as buttons for going back, volume regulation, turning on and off the device.

It can be integrated with testing framework TestNG. In this case UI Automator can generate informative and detailed reports, similar to reports generated by Ranorex. This tool also searches elements very fast.

Software testing experts find UI Automator good for quality mobile application testing on many Android platforms. It is one of the best tools for Android applications, as it is elaborated by Google specially for this operating system.

Usually about 80% of new software bugs reproduce on all the supported platforms. So, one may perform mobile testing on one of widely used platforms and discover up to 80% of defects. The rest 20% will be found on all the other platforms. That means that in most cases it is better to test software product thoroughly on fewer platforms than hastily on numerous.

At present Android 4.1 is installed on about 66% of all devices using Android operating system. That is why many automated test engineers often decide that UI Automator is the most suitable solution.

Website Cookie Testing

What is Cookie?
Cookie is small information stored in text file on user’s hard drive by web server. This information is later used by web browser to retrieve information from that machine. Generally cookie contains personalized user data or information that is used to communicate between different web pages.

Why Cookies are used?
Cookies are nothing but the user’s identity and used to track where the user navigated throughout the web site pages. The communication between web browser and web server is stateless.

For example if you are accessing domain http://www.example.com/1.html then web browser will simply query to example.com web server for the page 1.html. Next time if you type page as http://www.example.com/2.html then new request is send to example.com web server for sending 2.html page and web server don’t know anything about to whom the previous page 1.html served.

What if you want the previous history of this user communication with the web server? You need to maintain the user state and interaction between web browser and web server somewhere. This is where cookie comes into picture. Cookies serve the purpose of maintaining the user interactions with web server.

How cookies work?
The HTTP protocol used to exchange information files on the web is used to maintain the cookies. There are two types of HTTP protocol. Stateless HTTP and Stateful HTTP protocol. Stateless HTTP protocol does not keep any record of previously accessed web page history. While Stateful HTTP protocol do keep some history of previous web browser and web server interactions and this protocol is used by cookies to maintain the user interactions.

Whenever user visits the site or page that is using cookie, small code inside that HTML page (Generally a call to some language script to write the cookie like cookies in JAVAScript, PHP, Perl) writes a text file on users machine called cookie.
Here is one example of the code that is used to write cookie and can be placed inside any HTML page:

Set-Cookie: NAME=VALUE; expires=DATE; path=PATH; domain=DOMAIN_NAME;

When user visits the same page or domain later time this cookie is read from disk and used to identify the second visit of the same user on that domain. Expiration time is set while writing the cookie. This time is decided by the application that is going to use the cookie.

Generally two types of cookies are written on user machine.

1) Session cookies: This cookie is active till the browser that invoked the cookie is open. When we close the browser this session cookie gets deleted. Some time session of say 20 minutes can be set to expire the cookie.
2) Persistent cookies: The cookies that are written permanently on user machine and lasts for months or years.

Where cookies are stored?
When any web page application writes cookie it get saved in a text file on user hard disk drive. The path where the cookies get stored depends on the browser. Different browsers store cookie in different paths. E.g. Internet explorer store cookies on path “C:\Documents and Settings\Default User\Cookies”
Here the “Default User” can be replaced by the current user you logged in as. Like “Administrator”, or user name like “Vijay” etc.
The cookie path can be easily found by navigating through the browser options. In Mozilla Firefox browser you can even see the cookies in browser options itself. Open the Mozila browser, click on Tools->Options->Privacy and then “Show cookies” button.

How cookies are stored?
Lets take example of cookie written by rediff.com on Mozilla Firefox browser:
On Mozilla Firefox browser when you open the page rediff.com or login to your rediffmail account, a cookie will get written on your Hard disk. To view this cookie simply click on “Show cookies” button mentioned on above path. Click on Rediff.com site under this cookie list. You can see different cookies written by rediff domain with different names.

Site: Rediff.com Cookie name: RMID
Name: RMID (Name of the cookie)
Content: 1d11c8ec44bf49e0… (Encrypted content)
Domain: .rediff.com
Path: / (Any path after the domain name)
Send For: Any type of connection
Expires: Thursday, December 31, 2020 11:59:59 PM

Applications where cookies can be used:

1) To implement shopping cart:
Cookies are used for maintaining online ordering system. Cookies remember what user wants to buy. What if user adds some products in their shopping cart and if due to some reason user don’t want to buy those products this time and closes the browser window? When next time same user visits the purchase page he can see all the products he added in shopping cart in his last visit.

2) Personalized sites:
When user visits certain pages they are asked which pages they don’t want to visit or display. User options are get stored in cookie and till the user is online, those pages are not shown to him.

3) User tracking: 
To track number of unique visitors online at particular time.

4) Marketing:
Some companies use cookies to display advertisements on user machines. Cookies control these advertisements. When and which advertisement should be shown? What is the interest of the user? Which keywords he searches on the site? All these things can be maintained using cookies.

5) User sessions:
Cookies can track user sessions to particular domain using user ID and password.

Drawbacks of cookies:

1) Even writing Cookie is a great way to maintain user interaction, if user has set browser options to warn before writing any cookie or disabled the cookies completely then site containing cookie will be completely disabled and can not perform any operation resulting in loss of site traffic.

2) Too many Cookies:
If you are writing too many cookies on every page navigation and if user has turned on option to warn before writing cookie, this could turn away user from your site.

3) Security issues:
Some times users personal information is stored in cookies and if someone hack the cookie then hacker can get access to your personal information. Even corrupted cookies can be read by different domains and lead to security issues.

4) Sensitive information:
Some sites may write and store your sensitive information in cookies, which should not be allowed due to privacy concerns.

This should be enough to know what cookies are. If you want more cookie info see Cookie Central page.

Some Major Test cases for web application cookie testing:

The first obvious test case is to test if your application is writing cookies properly on disk. You can use the Cookie Tester applicationalso if you don’t have any web application to test but you want to understand the cookie concept for testing.

Test cases: 

1) As a Cookie privacy policy make sure from your design documents that no personal or sensitive data is stored in the cookie.

2) If you have no option than saving sensitive data in cookie make sure data stored in cookie is stored in encrypted format.

3) Make sure that there is no overuse of cookies on your site under test. Overuse of cookies will annoy users if browser is prompting for cookies more often and this could result in loss of site traffic and eventually loss of business.

4) Disable the cookies from your browser settings: If you are using cookies on your site, your sites major functionality will not work by disabling the cookies. Then try to access the web site under test. Navigate through the site. See if appropriate messages are displayed to user like “For smooth functioning of this site make sure that cookies are enabled on your browser”. There should not be any page crash due to disabling the cookies. (Please make sure that you close all browsers, delete all previously written cookies before performing this test)

5) Accepts/Reject some cookies: The best way to check web site functionality is, not to accept all cookies. If you are writing 10 cookies in your web application then randomly accept some cookies say accept 5 and reject 5 cookies. For executing this test case you can set browser options to prompt whenever cookie is being written to disk. On this prompt window you can either accept or reject cookie. Try to access major functionality of web site. See if pages are getting crashed or data is getting corrupted.

6) Delete cookie: Allow site to write the cookies and then close all browsers and manually delete all cookies for web site under test. Access the web pages and check the behavior of the pages.

7) Corrupt the cookies: Corrupting cookie is easy. You know where cookies are stored. Manually edit the cookie in notepad and change the parameters to some vague values. Like alter the cookie content, Name of the cookie or expiry date of the cookie and see the site functionality. In some cases corrupted cookies allow to read the data inside it for any other domain. This should not happen in case of your web site cookies. Note that the cookies written by one domain say rediff.com can’t be accessed by other domain say yahoo.com unless and until the cookies are corrupted and someone trying to hack the cookie data.

8 ) Checking the deletion of cookies from your web application page: Some times cookie written by domain say rediff.com may be deleted by same domain but by different page under that domain. This is the general case if you are testing some ‘action tracking’ web portal. Action tracking or purchase tracking pixel is placed on the action web page and when any action or purchase occurs by user the cookie written on disk get deleted to avoid multiple action logging from same cookie. Check if reaching to your action or purchase page deletes the cookie properly and no more invalid actions or purchase get logged from same user.

9) Cookie Testing on Multiple browsers: This is the important case to check if your web application page is writing the cookies properly on different browsers as intended and site works properly using these cookies. You can test your web application on Major used browsers like Internet explorer (Various versions), Mozilla Firefox, Netscape, Opera etc.

10) If your web application is using cookies to maintain the logging state of any user then log in to your web application using some username and password. In many cases you can see the logged in user ID parameter directly in browser address bar. Change this parameter to different value say if previous user ID is 100 then make it 101 and press enter. The proper access message should be displayed to user and user should not be able to see other users account.

SQL Injection – How to Test Web Applications against SQL Injection Attacks

Security testing of web applications against SQL Injection, explained with simple examples – By Inder P Singh.

Many applications use some type of a database. An application under test might have a user interface that accepts user input that is used to perform the following tasks:

1.    Show the relevant stored data to the user e.g. the application checks the credentials of the user using the log in information entered by the user and exposes only the relevant functionality and data to the user

2.    Save the data entered by the user to the database e.g. once the user fills up a form and submits it, the application proceeds to save the data to the database; this data is then made available to the user in the same session as well as in subsequent sessions

Some of the user inputs might be used in framing SQL statements that are then executed by the application on the database. It is possible for an application NOT to handle the inputs given by the user properly. If this is the case, a malicious user could provide unexpected inputs to the application that are then used to frame and execute SQL statements on the database. This is called SQL injection. The consequences of such an action could be alarming.

The following things might result from SQL injection:

1. The user could log in to the application as another user, even as an administrator.

2. The user could view private information belonging to other users e.g. details of other users’ profiles, their transaction details etc.

3. The user could change application configuration information and the data of the other users.

4. The user could modify the structure of the database; even delete tables in the application database.

5. The user could take control of the database server and execute commands on it at will.

Since the consequences of allowing the SQL injection technique could be severe, it follows that SQL injection should be tested during the security testing of an application. Now with an overview of the SQL injection technique, let us understand a few practical examples of SQL injection.

Important: The SQL injection problem should be tested only in the test environment.

If the application has a log in page, it is possible that the application uses a dynamic SQL such as statement below. This statement is expected to return at least a single row with the user details from the Users table as the result set when there is a row with the user name and password entered in the SQL statement.

SELECT * FROM Users WHERE User_Name = ‘” & strUserName & “‘ AND Password = ‘” & strPassword & “’;”

If the tester would enter John as the strUserName (in the textbox for user name) and Smith as strPassword (in the textbox for password), the above SQL statement would become:

SELECT * FROM Users WHERE User_Name = ‘John’ AND Password = ‘Smith’;

If the tester would enter John’– as strUserName and no strPassword, the SQL statement would become:

SELECT * FROM Users WHERE User_Name = ‘John’– AND Password = ‘Smith’;

Note that the part of the SQL statement after John is turned into a comment. If there were any user with the user name of John in the Users table, the application could allow the tester to log in as the user John. The tester could now view the private information of the user John.

What if the tester does not know the name of any existing user of the application? In such a case, the tester could try common user names like admin, administrator and sysadmin. If none of these users exist in the database, the tester could enter John’ or ‘x’=’x as strUserName and Smith’ or ‘x’=’x  as strPassword. This would cause the SQL statement to become like the one below.

SELECT * FROM Users WHERE User_Name = ‘John’ or ‘x’=’x’ AND Password = ‘Smith’ or ‘x’=’x’;

Since ‘x’=’x’ condition is always true, the result set would consist of all the rows in the Users table. The application could allow the tester to log in as the first user in the Users table.

Important: The tester should request the database administrator or the developer to copy the table in question before attempting the following SQL injection.

If the tester would enter John’; DROP table users_details;’—as strUserName and anything as strPassword, the SQL statement would become like the one below.

SELECT * FROM Users WHERE User_Name = ‘John’; DROP table users_details;’ –‘ AND Password = ‘Smith';

This statement could cause the table “users_details” to be permanently deleted from the database.

Though the above examples deal with using the SQL injection technique only the log in page, the tester should test this technique on all the pages of the application that accept user input in textual format e.g. search pages, feedback pages etc.

SQL injection might be possible in applications that use SSL. Even a firewall might not be able to protect the application against the SQL injection technique.

I have tried to explain the SQL injection technique in a simple form. I would like to re-iterate that SQL injection should be tested only in a test environment and not in the development environment, production environment or any other environment. Instead of manually testing whether the application is vulnerable to SQL injection or not, one could use a web vulnerability scanner that checks for SQL injection.

Penetration Testing

What is Penetration Testing?
It’s the process to identify security vulnerabilities in an application by evaluating the system or network with various malicious techniques. Purpose of this test is to secure important data from outsiders like hackers who can have unauthorized access to system. Once vulnerability is identified it is used to exploit system in order to gain access to sensitive information.

Causes of vulnerabilities:
– Design and development errors
– Poor system configuration
– Human errors

Why Penetration testing?

– Financial data must be secured while transferring between different systems
– Many clients are asking for pen testing as part of the software release cycle
– To secure user data
– To find security vulnerabilities in an application

It’s very important for any organization to identify security issues present in internal network and computers. Using this information organization can plan defense against any hacking attempt. User privacy and data security are the biggest concerns nowadays. Imagine if any hacker manage to get user details of social networking site like Facebook. Organization can face legal issues due to a small loophole left in a software system. Hence big organizations are looking for PCI compliance certifications before doing any business with third party clients.

What should be tested?
– Software
– Hardware
– Network
– Process

Penetration Testing Types:

1) Social Engineering: Human errors are the main causes of security vulnerability. Security standards and policies should be followed by all staff members to avoid social engineering penetration attempt. Example of these standards include not to mention any sensitive information in email or phone communication. Security audits can be conducted to identify and correct process flaws.

2) Application Security Testing: Using software methods one can verify if the system is exposed to security vulnerabilities.

3) Physical Penetration Test: Strong physical security methods are applied to protect sensitive data. This is generally useful in military and government facilities. All physical network devices and access points are tested for possibilities of any security breach.

Pen Testing Techniques:
1) Manual penetration test
2) Using automated penetration test tools
3) Combination of both manual and automated process
The third process is more common to identify all kinds of vulnerabilities.

Penetration Testing Tools:

Automated tools can be used to identify some standard vulnerability present in an application. Pentest tools scan code to check if there is malicious code present which can lead to potential security breach. Pentest tools can verify security loopholes present in the system like data encryption techniques and hard coded values like username and password.

Criteria to select the best penetration tool:
– It should be easy to deploy, configure and use.
– It should scan your system easily.
– It should categorize vulnerabilities based on severity that needs immediate fix.
– It should be able to automate verification of vulnerabilities.
– It should re-verify exploits found previously.
– It should generate detailed vulnerability reports and logs.

Once you know what tests you need to perform you can either train your internal test resources or hire expert consultants to do the penetration task for you.

Examples of Free and Commercial Tools –
Nmap, Nessus, Metasploit, Wireshark, OpenSSL, Cain & Abel, THC Hydra, w3af
Commercial services: Pure Hacking, Torrid Networks, SecPoint,Veracode.

Limitations of Pentest tools: Sometimes these tools can flag false positive output which results in spending more developer time on analyzing such vulnerabilities which are not present.

Manual Penetration Test:

It’s difficult to find all vulnerabilities using automated tools. There are some vulnerabilities which can be identified by manual scan only. Penetration testers can perform better attacks on application based on their skills and knowledge of system being penetrated. The methods like social engineering can be done by humans only. Manual checking includes design, business logic as well as code verification.

Penetration Test Process:
Let’s discuss the actual process followed by test agencies or penetration testers. Identifying vulnerabilities present in system is the first important step in this process. Corrective action is taken on these vulnerability and same penetration tests are repeated until system is negative to all those tests.

We can categorize this process in following methods:
1) Data collection: Various methods including Google search are used to get target system data. One can also use web page source code analysis technique to get more info about the system, software and plugin versions. There are many free tools and services available in the market which can give you information like database or table names, DB versions, software versions, hardware used and various third party plugins used in the target system.

2) Vulnerability Assessment: Based on the data collected in first step one can find the security weakness in the target system. This helps penetration testers to launch attacks using identified entry points in the system.

3) Actual Exploit: This is crucial step. It requires special skills and techniques to launch attack on target system. Experienced penetration testers can use their skills to launch attack on the system.

4) Result analysis and report preparation: After completion of penetration tests detailed reports are prepared for taking corrective actions. All identified vulnerabilities and recommended corrective methods are listed in these reports. You can customize vulnerability report format (HTML, XML, MS Word or PDF) as per your organization needs.

Penetration testing sample test cases (test scenarios):

Remember this is not functional testing. In Pentest your goal is to find security holes in the system. Below are some generic test cases and not necessarily applicable for all applications.

1) Check if web application is able to identify spam attacks on contact forms used in the website.
2) Proxy server – Check if network traffic is monitored by proxy appliances. Proxy server make it difficult for hackers to get internal details of the network thus protecting the system from external attacks.
3) Spam email filters – Verify if incoming and outgoing email traffic is filtered and unsolicited  emails are blocked. Many email clients come with in-build spam filters which needs to be configured as per your needs. These configuration rules can be applied on email headers, subject or body.
4) Firewall – Make sure entire network or computers are protected with Firewall. Firewall can be a software or hardware to block unauthorized access to system. Firewall can prevent sending data outside the network without your permission.
5) Try to exploit all servers, desktop systems, printers and network devices.
6) Verify that all usernames and passwords are encrypted and transferred over secured connection like https.
7) Verify information stored in website cookies. It should not be in readable format.
8 ) Verify previously found vulnerabilities to check if the fix is working.
9) Verify if there is no open port in network.
11) Verify all telephone devices.
12) Verify WIFI network security.
13) Verify all HTTP methods. PUT and Delete methods should not be enabled on web server .
14) Password should be at least 8 character long containing at least one number and one special character.
15) Username should not be like “admin” or “administrator”.
16) Application login page should be locked upon few unsuccessful login attempts.
17) Error messages should be generic and should not mention specific error details like “Invalid username” or “Invalid password”.
19) Verify if special characters, html tags and scripts are handled properly as an input value.
20) Internal system details should not be revealed in any of the error or alert messages.
21) Custom error messages should be displayed to end user in case of web page crash.
22) Verify use of registry entries. Sensitive information should not be kept in registry.
23) All files must be scanned before uploading to server.
24) Sensitive data should not be passed in urls while communicating with different internal modules of the web application.
25) There should not be any hard coded username or password in the system.
26) Verify all input fields with long input string with and without spaces.
27) Verify if reset password functionality is secure.
28) Verify application for SQL Injection.
29) Verify application for Cross Site Scripting.
31) Important input validations should be done at server side instead of JavaScript checks at client side.
32) Critical resources in the system should be available to authorized persons and services only.
33) All access logs should be maintained with proper access permissions.
34) Verify user session ends upon log off.
35) Verify that directory browsing is disabled on server.
36) Verify that all applications and database versions are up to date.
37) Verify url manipulation to check if web application is not showing any unwanted information.
38) Verify memory leak and buffer overflow.
39) Verify if incoming network traffic is scanned to find Trojan attacks.
40) Verify if system is safe from Brute Force Attacks – a trial and error method to find sensitive information like passwords.
41) Verify if system or network is secured from DoS (denial-of-service) attacks. Hacker can target network or single computer with continuous requests due to which resources on target system gets overloaded resulting in denial of service for legit requests.

These are just the basic test scenarios to get started with Pentest. There are hundreds of advanced penetration methods which can be done either manually or with the help of automation tools.

Pen Testing Standards – PCI DSS (Payment Card Industry Data Security Standard), OWASP (Open Web Application Security Project),ISO/IEC 27002, OSSTMM (The Open Source Security Testing Methodology Manual).
Certifications – GPEN, Associate Security Tester (AST), Senior Security Tester (SST), Certified Penetration Tester (CPT).