Continous Integration

1. Tools - click here

2. CI - Tool - Cruise control - click here

CI Tool - Cruise Control

Installation

1. downloaded cruisecontrol from openlogic site with version 2.8.2

2. Run the exe file, stored in C:\cruisecontrol path.

3. After successful completion, navigate to C:\cuisecontrol folder, & check all cruisecontrol.bat, wrapper.exe exists.

4. Before kick start, just is cruise installed correctly or not, by invoking service on browser url as:

    http://localhost:8080/dashboard

    U should get DASHBOARD, then it is correct!!


5.

Continuous Integration ( CI ) Tools

Continuous Integration ( CI ) Tools

1. Hudson

Hudson is a continuous integration tool written in Java, which runs in a servlet container, such as Apache Tomcat or the GlassFish application server. It supports SCM tools including CVS, Subversion, Git and Clearcase and can execute Apache Ant and Apache Maven based projects, as well as arbitrary shell scripts and Windows batch commands.

2. CruiseControl NET

CruiseControl.NET is an Automated Continuous Integration server, implemented using the Microsoft .NET Framework. 

3. Apache Continuum

Apache Continuum is an enterprise-ready continuous integration server with features such as automated builds, release management, role-based security, and integration with popular build tools and source control management systems. Whether you have a centralized build team or want to put control of releases in the hands of developers, Continuum can help you improve quality and maintain a consistent build environment.

4. Draco NET

Draco.NET is a Windows service application designed to facilitate continuous integration. Draco.NET monitors your source code repository, automatically rebuilds your project when changes are detected and then emails you the build result along with a list of changes since the last build.

5. CABIE

Continuous Automated Build and Integration Environment. Cabie is a multi-platform, multi-cm client/server based application providing both command line and web-based access to real time build monitoring and execution information. Cabie builds jobs based upon configuration information stored in MySQL and will support virtually any build that can be called from the command line. Cabie provides a centralized collection point for all builds providing web based dynamic access, the collector is SQL ba

6. Buildbot

The BuildBot is a system to automate the compile test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everyw

7. Beebox

Beebox is a complete solution for any development and systems management team. It allows you to standardize your development, deployment and quality assurance processes, automate repetitive tasks, and greatly simplify teamwork in a continuous integration environment.

8. Apache Gump

Apache Gump is an open source Continuous Integration system, which aims to build and test all the open source Java projects, every night. Its aim is to make sure that all the projects are compatible, at both the API level and in terms of functionality matching specifications. It is hosted at gump.apache.org, and runs every night on the official Sun JVM.

9. ControlTier

ControlTier is an open source, cross-platform build and deployment automation framework. ControlTier can help you to coordinate and scale service management and administration activities across multiple nodes and application tiers.

10. CruiseControl

CruiseControl is both a continuous integration tool and an extensible framework for creating a custom continuous build process. It includes dozens of plugins for a variety of source controls, build technologies, and notifications schemes including email and instant messaging. A web interface provides details of the current and previous builds. And the standard CruiseControl distribution is augmented through a rich selection of 3rd Party Tools.

A

Apache Continuum
Apache Continuum is an enterprise-ready continuous integration server with features such as automated builds, release management, role-based security, and integration with popular build tools and source control management systems. Whether you have a centralized build team or want to put control of releases in the hands of developers, Continuum can help you improve quality and maintain a consistent build environment.

Apache Gump
Gump is Apache's continuous integration tool. It is written in python and fully supports Apache Ant, Apache Maven (1.x and 2.x) and other build tools. Gump is unique in that it builds and compiles software against the latest development versions of those projects. This allows gump to detect potentially incompatible changes to that software just a few hours after those changes are checked into the version control system. Notifications are sent to the project team as soon as such a change is detected, referencing more detailed reports available online.

B

Beebox
beebox is a complete solution for any development and systems management team. It allows you to standardize your development, deployment and quality assurance processes, automate repetitive tasks, and greatly simplify teamwork in a continuous integration environment.

Bitten
Bitten is a Python-based framework for collecting various software metrics via continuous integration. It builds on Trac to provide an integrated web-based user interface.

BuildBot
The BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure. The guilty developer can be identified and harassed without human intervention. By running the builds on a variety of platforms, developers who do not have the facilities to test their changes everywhere before checkin will at least know shortly afterwards whether they have broken the build or not. Warning counts, lint checks, image size, compile time, and other build parameters can be tracked over time, are more visible, and are therefore easier to improve.

C

CABIE
Continuous Automated Build and Integration Environment. Cabie is a multi-platform, multi-cm client/server based application providing both command line and web-based access to real time build monitoring and execution information. Cabie builds jobs based upon configuration information stored in MySQL and will support virtually any build that can be called from the command line. Cabie provides a centralized collection point for all builds providing web based dynamic access, the collector is SQL based and provides information for all projects under Cabie's control. Cabie can be integrated with bug tracking systems and test systems with some effort depending on the complexity of those systems. With the idea in mind that most companies create build systems from the ground up, Cabie was designed to not have to re-write scripted builds but instead to integrate existing build scripts into a smart collector. Cabie provides rapid email notification and RSS integration to quickly handle build issues. Cabie provides the ability to run builds in parallel, series, to poll jobs or to allow the use of scripted nightly builds. Cabie is perfect for agile development in an environment that requires multiple languages and tools. Cabie supports Perforce, Subversion and CVS. The use of a backend broker allows anyone with perl skills to write support for additional CM systems.

Cerberus
Cerberus is a lightweight and easy-to-use Continuous Builder software for Ruby. It could be run periodically from a scheduler and check if application tests are broken. In case of failed tests Cerberus sends notification to developers.

Control Tier
ControlTier is a community driven, cross-platform software system used to coordinate application service management activities across multiple nodes and application tiers. The project is fully open source and many of the project contributions come from DTO Solutions ongoing consulting work for large scale e-commerce, software-as-a-service, and financial services operations.

Sponsor: Zed Builds and Bugs
A Java based Continuous Integration Server that combines Continuous Integration, Bug Tracking, Discussion Forums, and Development Wiki.

CruiseControl
CruiseControl is both a continuous integration tool and an extensible framework for creating a custom continuous build process. It includes dozens of plugins for a variety of source controls, build technologies, and notifications schemes including email and instant messaging. A web interface provides details of the current and previous builds. And the standard CruiseControl distribution is augmented through a rich selection of 3rd Party Tools.

CruiseControl.NET
CruiseControl.NET (CCNet) consists of a suite of applications, but at its core is the CruiseControl.NET Server which is an automated integration server. The Server automates the integration process by monitoring the team's source control repository directly. Every time a developer commits a new set of modifications, the server will automatically launch an integration build to validate the changes. When the build is complete, the server notifies the developer whether the changes that they committed integrated successfully or not.

CruiseControl.rb
CruiseControl.rb is a continuous integration tool. Its basic purpose in life is to alert members of a software project when one of them checks something into source control that breaks the build. CC.rb is easy to install, pleasant to use and simple to hack. It's written in Ruby.

CDash
CDash is an open source, web-based software testing server. CDash aggregates, analyzes and displays the results of software testing processes submitted from clients located around the world. Developers depend on CDash to convey the state of a software system, and to continually improve its quality. CDash is a part of a larger software process that integrates Kitware's CMake, CTest, and CPack tools, as well as other external packages used to design, manage and maintain large-scale software systems.

D

Draco.NET
Draco.NET is a Windows service application designed to facilitate continuous integration. Draco.NET monitors your source code repository, automatically rebuilds your project when changes are detected and then emails you the build result along with a list of changes since the last build.

H

Sponsor: Zed Builds and Bugs
A Java based Continuous Integration Server that combines Continuous Integration, Bug Tracking, Discussion Forums, and Development Wiki.

Hudson
Hudson monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those things, current Hudson focuses on the following two jobs:

1.              Building/testing software projects continuously, just like CruiseControl or DamageControl. In a nutshell, Hudson provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. The automated, continuous build increases the productivity.

2.              Monitoring executions of externally-run jobs, such as cron jobs and procmail jobs, even those that are run on a remote machine. For example, with cron, all you receive is regular e-mails that capture the output, and it is up to you to look at them diligently and notice when it broke. Hudson keeps those outputs and makes it easy for you to notice when something is wrong.

J

Jhbuild
Jhbuild is a tool used to build the whole GNOME desktop from the git source, however, it can be used to build other projects creating a moduleset for it

L

LuntBuild
Luntbuild is a powerful build automation and management tool. Continuous Integration or nightly builds can be easily set using a clean web interface. Executed builds are well managed using functions such as search, categorization, promotion, patching, deletion, etc. It also acts as a central build artifacts repository and download area for your whole team.

S

Sin
Sin is a framework for implementing Continuous Integration on top of the Subversion version control system.

T

Tinderbox
Tinderbox is a webtool that Mozilla developers use to check whether the current source code compiles on various platforms and passes automated test suites.

Test Automation Strategy

Test Automation Strategy

Test Complexity

Test Run Occurrence

Reuse of existing Scripts

Product Stability

GUI Stability

Execution Time

Implementability

ROI /Metrics Reporting 
Evalution of Automation Tool 
Automation Framework

Hardware requirements (eg., CPU speed, RAM, network access, etc) 
Test Environment (OS, Automation Tool, Test Management tool, Virtualisation, etc) 
How to integrate with Continuous Build System tool 
Version control system 
Test result reporting 
Frequency of test runs 
Scope of the regression tests 
Resourcing (who will write the scripts, who will run the tests, who will maintain the scripts)

Ability to take backup of whole system/logs in case of failures. 
Ability to auto reinstall apps and re execute the failed test cases. 
Hosting test results. 
A README in each suite to figure out the regressions.

Project Overview. 
Business Risks. 
Testing Milestones. 
Testing Approach. 
Testing Environment

Server Side Testing

What is server side testing

Web server contains the application code. Verifying server side daemons & process using application programming language.  It's testing the applications and daemons that run on a server.  Server Side testing can involve testing of Servlets and Controllers. When the user requests from the front end, how the requests hits the server, which file is initiated first from the application code & how it behaves?? There are two types of logs.  Server / system related logs & application logs. Analyzing & verifying these logs under various conditions. A web is hosted on the server.  So hosting process & debugging is also very challenging for the server side testing. compatibility of server software, hard ware, network connections, database compatibility, external interface compatibility

Difference between Client side testing & Server side testing

Server side testing is valuing the clent parameters after reaching to server, where as clent side testing will do that client machine only.   Server is testing will be done thru application programming languages like java/jsp, where as client side testing has been taken care of Java Script.   Server side testing is required in below cases: 1: Required values from database to process/valuate the Client parameters. 2: Required to apply the business logic. 3: Required any type server side resources to check user resources.   Server side testing is required in below cases: 1: validating parameters.

                                                                                                                    

Server side testing - TOOLS

Apache CACTUS – Nice tool which is tied with JUnit.

Server Side Testing – Requirements or who can test this ???

A good technical person is the right person to test this.  Should have explored in the fields - network, servers, webhosting, debugging, installation, various OS (mainly linux), Good programming skills etc.,

Security Testing

Security Testing

Security Testing: Testing which confirms that the program can access to authorized personnel and that the authorized personnel can access the functions available to their security level. Security testing is testing how well the system is protected against unauthorized internal or external access, or willful damage. The purpose of security testing is to determine how well a system protects against unauthorized internal or external access or willful damage.

Types of Security Testing

1. Vulnerability Scanning 2. Security Scanning 3. Penetration Testing 4. Risk Assessment 5. Security Auditing 6. Ethical Hacking 7. Posture Assessment & Security Testing

Testing Sections

1. Information Security 2. Process Security 3. Internet Technology Security 4. Communications Security 5. Wireless Security 6. Physical Security

Vulnerability

Vulnerability Scanning is using automated software to scan one or more systems against known vulnerability signatures. Vulnerability analysis is a systematic review of networks and systems, that determines the adequacy of security measures, identifies security deficiencies, and evaluates the effectiveness of existing and planned safeguards. It justify the resources required to scope of organization's perimeter security or alternatively give you the piece of mind that your network is secure. Examples of this software are Nessus, Sara, and ISS.

Security

Security Scanning is a Vulnerability Scan plus Manual verification. The Security Analyst will then identify network weaknesses and perform a customized professional analysis.

Penetration

Penetration Testing takes a snapshot of the security on one machine, the "trophy". The Tester will attempt to gain access to the trophy and prove his access, usually, by saving a file on the machine. It is a controlled and coordinated test with the client to ensure that no laws are broken during the test. This is a live test mimicking the actions of real life attackers. Is the security of IT systems up to the task? Conducting a penetration test is a valuable experience in preparing your defenses against the real thing.

Risk Assessment

Risk Assessment involves a security analysis of interviews compiled with research of business, legal, and industry justifications.

Security Auditing

Security Auditing involves hands on internal inspection of Operating Systems and Applications, often via line-by-line inspection of the code. Thorough and frequent security audits will mean your network is more secure and less prone to attack.

Ethical Hacking

Ethical Hacking is basically a number of Penetration Tests on a number of systems on a network segment.  Posture Assessment & Security Testing combine Security Scanning, Ethical Hacking and Risk Assessments to show an overall Security Posture of the organization.

Security Testing tools

1. OWASP 2. SQL Map - click here to know more 3. SQL Dumper - click here to know more

OWASP

1. HTTP tool – Download zip > unzip > run gui file

Server Monitoring Tools ( SUT)

Performance Web Server Monitoring Tools

OPMON - http://www.crestechglobal.com/opmon.html

Nagios - http://www.nagios.org/

Hyperic - http://www.hyperic.com/

Opensmart - http://opensmart.sourceforge.net/

Performance Database Server Monitoring Tools

MySQL - Mtop

MySQL - http://phpmytop.sourceforge.net/

For all - http://www.dbtuna.com

performance vs load vs stress

Performance vs. load vs. stress testing

Performance testing The goal of performance testing is not to find bugs, but to eliminate bottlenecks and establish a baseline for future regression testing. To conduct performance testing is to engage in a carefully controlled process of measurement and analysis. Ideally, the software under test is already stable enough so that this process can proceed smoothly. A clearly defined set of expectations is essential for meaningful performance testing. If you don't know where you want to go in terms of the performance of the system, then it matters little which direction you take (remember Alice and the Cheshire Cat?). For example, for a Web application, you need to know at least two things: ·                     expected load in terms of concurrent users or HTTP connections ·                     acceptable response time Once you know where you want to be, you can start on your way there by constantly increasing the load on the system while looking for bottlenecks. To take again the example of a Web application, these bottlenecks can exist at multiple levels, and to pinpoint them you can use a variety of tools: ·                     at the application level, developers can use profilers to spot inefficiencies in their code (for example poor search algorithms) ·                     at the database level, developers and DBAs can use database-specific profilers and query optimizers ·                     at the operating system level, system engineers can use utilities such as top, vmstat, iostat (on Unix-type systems) and PerfMon (on Windows) to monitor hardware resources such as CPU, memory, swap, disk I/O; specialized kernel monitoring software can also be used ·                     at the network level, network engineers can use packet sniffers such as tcpdump, network protocol analyzers such as ethereal, and various utilities such as netstat, MRTG, ntop, mii-tool From a testing point of view, the activities described above all take a white-box approach, where the system is inspected and monitored "from the inside out" and from a variety of angles. Measurements are taken and analyzed, and as a result, tuning is done. However, testers also take a black-box approach in running the load tests against the system under test. For a Web application, testers will use tools that simulate concurrent users/HTTP connections and measure response times. Some lightweight open source tools I've used in the past for this purpose are ab, siege, httperf. A more heavyweight tool I haven't used yet is OpenSTA. I also haven't used The Grinder yet, but it is high on my TODO list. When the results of the load test indicate that performance of the system does not meet its expected goals, it is time for tuning, starting with the application and the database. You want to make sure your code runs as efficiently as possible and your database is optimized on a given OS/hardware configurations. TDD practitioners will find very useful in this context a framework such as Mike Clark's jUnitPerf, which enhances existing unit test code with load test and timed test functionality. Once a particular function or method has been profiled and tuned, developers can then wrap its unit tests in jUnitPerf and ensure that it meets performance requirements of load and timing. Mike Clark calls this "continuous performance testing". I should also mention that I've done an initial port of jUnitPerf to Python -- I called it pyUnitPerf. If, after tuning the application and the database, the system still doesn't meet its expected goals in terms of performance, a wide array of tuning procedures is available at the all the levels discussed before. Here are some examples of things you can do to enhance the performance of a Web application outside of the application code per se: ·                     Use Web cache mechanisms, such as the one provided by Squid ·                     Publish highly-requested Web pages statically, so that they don't hit the database ·                     Scale the Web server farm horizontally via load balancing ·                     Scale the database servers horizontally and split them into read/write servers and read-only servers, then load balance the read-only servers ·                     Scale the Web and database servers vertically, by adding more hardware resources (CPU, RAM, disks) ·                     Increase the available network bandwidth Performance tuning can sometimes be more art than science, due to the sheer complexity of the systems involved in a modern Web application. Care must be taken to modify one variable at a time and redo the measurements, otherwise multiple changes can have subtle interactions that are hard to qualify and repeat. In a standard test environment such as a test lab, it will not always be possible to replicate the production server configuration. In such cases, a staging environment is used which is a subset of the production environment. The expected performance of the system needs to be scaled down accordingly. The cycle "run load test->measure performance->tune system" is repeated until the system under test achieves the expected levels of performance. At this point, testers have a baseline for how the system behaves under normal conditions. This baseline can then be used in regression tests to gauge how well a new version of the software performs. Another common goal of performance testing is to establish benchmark numbers for the system under test. There are many industry-standard benchmarks such as the ones published by TPC, and many hardware/software vendors will fine-tune their systems in such ways as to obtain a high ranking in the TCP top-tens. It is common knowledge that one needs to be wary of any performance claims that do not include a detailed specification of all the hardware and software configurations that were used in that particular test.

Load testing We have already seen load testing as part of the process of performance testing and tuning. In that context, it meant constantly increasing the load on the system via automated tools. For a Web application, the load is defined in terms of concurrent users or HTTP connections. In the testing literature, the term "load testing" is usually defined as the process of exercising the system under test by feeding it the largest tasks it can operate with. Load testing is sometimes called volume testing, or longevity/endurance testing. Examples of volume testing: ·                     testing a word processor by editing a very large document ·                     testing a printer by sending it a very large job ·                     testing a mail server with thousands of users mailboxes ·                     a specific case of volume testing is zero-volume testing, where the system is fed empty tasks Examples of longevity/endurance testing: ·                     testing a client-server application by running the client in a loop against the server over an extended period of time Goals of load testing: ·                     expose bugs that do not surface in cursory testing, such as memory management bugs, memory leaks, buffer overflows, etc. ·                     ensure that the application meets the performance baseline established during performance testing. This is done by running regression tests against the application at a specified maximum load. Although performance testing and load testing can seem similar, their goals are different. On one hand, performance testing uses load testing techniques and tools for measurement and benchmarking purposes and uses various load levels. On the other hand, load testing operates at a predefined load level, usually the highest load that the system can accept while still functioning properly. Note that load testing does not aim to break the system by overwhelming it, but instead tries to keep the system constantly humming like a well-oiled machine. In the context of load testing, I want to emphasize the extreme importance of having large datasets available for testing. In my experience, many important bugs simply do not surface unless you deal with very large entities such thousands of users in repositories such as LDAP/NIS/Active Directory, thousands of mail server mailboxes, multi-gigabyte tables in databases, deep file/directory hierarchies on file systems, etc. Testers obviously need automated tools to generate these large data sets, but fortunately any good scripting language worth its salt will do the job.

Stress testing Stress testing tries to break the system under test by overwhelming its resources or by taking resources away from it (in which case it is sometimes called negative testing). The main purpose behind this madness is to make sure that the system fails and recovers gracefully -- this quality is known as recoverability. Where performance testing demands a controlled environment and repeatable measurements, stress testing joyfully induces chaos and unpredictability. To take again the example of a Web application, here are some ways in which stress can be applied to the system: ·                     double the baseline number for concurrent users/HTTP connections ·                     randomly shut down and restart ports on the network switches/routers that connect the servers (via SNMP commands for example) ·                     take the database offline, then restart it ·                     rebuild a RAID array while the system is running ·                     run processes that consume resources (CPU, memory, disk, network) on the Web and database servers I'm sure devious testers can enhance this list with their favorite ways of breaking systems. However, stress testing does not break the system purely for the pleasure of breaking it, but instead it allows testers to observe how the system reacts to failure. Does it save its state or does it crash suddenly? Does it just hang and freeze or does it fail gracefully? On restart, is it able to recover from the last good state? Does it print out meaningful error messages to the user, or does it merely display incomprehensible hex codes? Is the security of the system compromised because of unexpected failures? And the list goes on.