Testing Methodologies

Coding: The Art of Construction

Testing: The Art of Destruction. An integral part of Quality Assurance.

Productive QA: Involves activities that improve the product.

Analytical QA: Compromises activities which ascertain its level of Quality. Software Testing comes under Analytical QA.


Software Testing:

1. Testing involves operation of a system or application under controlled conditions and evaluating the results.

2. Every Test consists of 3 steps :-

Planning : Inputs to be given, results to be obtained and the process to proceed is to planned.

Execution : preparing test environment, Completing the test, and determining test results.

Evaluation : compare the actual test outcome with what the correct outcome should have been.


Pareto principle’ - says that 80% of the errors are concentrated at 20% of the code only – forms a good criteria for testing.

Classifications in Testing:

Various methodologies of testing exist which can best be classified into 2 major categories:

Testing Methods:

Black Box Testing:

Comparison Testing

Graph Based Testing

Boundary Value Testing

Equivalence class Testing

Gray Box Testing:

Similar to Black box but the test cases, risk assessments, and test methods involved in gray box testing are developed based on the knowledge of the internal data and flow structures.

White Box Testing:

Mutation Testing

Basic Path Testing

Control Structure Testing

Testing Levels:

Unit Testing

Integration Testing

System Testing

Acceptance Testing


Black Box Testing:

Also called ‘Functional Testing’ as it concentrates on testing of the functionality rather than the internal details of code.

Test cases are designed based on the task descriptions.

Equivalence Class Testing: Test inputs are classified into Equivalence classes such that one input check validates all the input values in that class.

Boundary Value Testing: Boundary values of the Equivalence classes are considered and tested as they generally fail in Equivalence class testing.

Comparison Testing: Test cases results are compared with the features/results of other products e.g. competitor product.

Graph based Testing: Cause and effect graphs are generated and cyclometric complexity considered in using the test cases.


White Box Testing:

Also called ‘Structural Testing / Glass Box Testing’ is used for testing the code keeping the system specs in mind.

Inner working is considered and thus Developers Test.

Mutation Testing: Number of mutants of the same program created with minor changes and none of their result should coincide with that of the result of the original program given same test case.

Basic Path Testing: Testing is done based on Flow graph notation, uses Cyclometric complexity & Graph matrices.

Control Structure Testing: The Flow of control execution path is considered for testing. It does also checks:-

Conditional Testing: Branch Testing, Domain Testing.

Data Flow Testing.

Loop testing: Simple, Nested, Conditional, and Unstructured Loops



Unit Testing:

Unit Testing is primarily carried out by the developers themselves.

Deals functional correctness and the completeness of individual program units.

White box testing methods are employed.


Integration Testing:

Deals with testing when several program units are integrated.

Regression testing: Change of behavior due to modification or addition is called ‘Regression’. Used to bring changes from worst to least.

Incremental Integration Testing: Checks out for bugs which encounter when a module has been integrated to the existing.

Smoke Testing: It is the battery of test which checks the basic functionality of program. If fails then the program is not sent for further testing.


System Testing:

Deals with testing the whole program system for its intended purpose.

Recovery testing: System is forced to fail and is checked out how well the system recovers the failure.

Security Testing: Checks the capability of system to defend itself from hostile attack on programs and data.

Load & Stress Testing: The system is tested for max load and extreme stress points are figured out.

Performance Testing: Used to determine the processing speed.

Installation Testing: Installation & uninstallation is checked out in the target platform.


Acceptance Testing:

UAT ensures that the project satisfies the customer requirements.

Alpha Testing: It is the test done by the client at the developer’s site.

Beta Testing: This is the test done by the end-users at the client’s site.

Long Term Testing: Checks out for faults occurrence in a long term usage of the product.

Compatibility Testing: Testing to ensure compatibility of an application or Web site with different browsers, OSs, and hardware platforms.


For more software testing definitions, please go here

Software Development Life Cycle Models

Description of a process typically showing the main stages in producing and maintaining:
–Executable software (software development life cycle)
–Computer Aided Business Systems (systems development life cycle)

Basic Systems/Software Development Steps:

Define the problem… essential first step (initiating systems development)
–What is the problem?
–Work: compile problem statement or vision statement. Define objectives. Define Scope.
–Output: terms of reference for continuation

Feasibility (systems investigation)
–Expand on problem. Is there a feasible solution? How do we develop a new system?
–Work: High level analysis. Cost Benefit analysis. Organisational Feasibility. Technology available. Development and implementation plan (high level).
–Output: Feasibility report. Summary of preliminary findings and recommendations for next step.

Systems Analysis
–Understand what is done now. What are the requirements of a new system.
–Work: Analyse existing system’s ability to meet user’s needs. Determine what problems are, what the requirements are.
–Output: System model. Statement of requirements

Systems Design (two steps)
–How in general can the system be designed and realised
–Work: transform the requirements into a logical design. Prepare designs for various options.
–Output: alternative design options

Detailed Design
–How specifically should the system be designed and implemented
–Work: prepare a detailed physical design (architecture, hardware, data organisation, detailed design software procedures) for the chosen solution
–Output: detailed design specification

Implementation
–Realise the new system (put it into operation)
–Work: buy or assemble or develop programs. Write procedures. Set up organisation. Install hardware. Test. Convert from old system. Go live.
–Output: an operational system

Operation, Maintenance & Review
–Use the system, evaluate, modify if necessary, keep system in operation.
–Work: day to day running, modifications and maintenance
–Output: an ongoing operational system



Implementation of the SDLC

-“Traditional approach,” also known as the Waterfall Model
-Rapid Prototyping
-Incremental Approach
-Extreme Programming


Waterfall Model Advantages

By far the most commonly used approach up to now
All the most widely used methods are underpinned by the waterfall model
The stages are clear cut

Activities -> Outputs
Requirements Analysis -> Feasibility Study, Outline Requirements
Requirements Definition -> Requirements Specification
System Specification -> Functional Specification, Acceptance Test Specification, Draft user manual
Architectural Design -> Design architecture specification, System test specification
Interface Design -> Interface specification, Integration test specification
Detailed Design -> Design Specification, Unit test specification
Coding -> Program Code
Unit Testing -> Unit test result report
Module Testing -> Module test result report
Integration Testing -> Integration test report, Final user manual
System Testing -> System test report
Acceptance Testing -> Final System

Waterfall Disadvantages
Can be too rigid
One phase must be completed before proceeding on to the next
Difficult to incorporate user feedback without implementation

Rapid Prototyping



The prototype is an executable version of the functionality and interfaces (sometimes called a behavioural prototype)
Minimal time investment in prototypes
Facilitates user understanding and specification role
The prototype is a throwaway program and used to elicit and specify requirements (non-operational)
The system is then designed, constructed, and implemented using waterfall model
Rapid prototyping has been enabled by the availability of modern programming tools (screen painters, 4GLs, Database software, etc.)

Incremental



Planning
Management Cycle Plan: resource, execute, control
Planning means “thinking of everything”
Without a plan (defining some goal) it is impossible to evaluate how you are progressing toward that goal

Gantt Charts for Planning


Your Project

The SDLC gives you the stages you must follow to develop your program/system
You must plan how long you are going to take at each stage and what you are going to produce at each stage
Then you can evaluate how you are progressing (control)

Conclusion

There is a recognised systematic approach to software development
The recognised steps are:
Feasibility (investigation)
Analysis
Design
Implementation (including testing)
Maintenance and review
Each stage delivers outputs that progressively build the system
You can plan by assessing what you are going to deliver at each stage and how long you think it will take. You control, progress against the plan