Software testing life cycle

Although variations exist between organizations, there is a typical cycle for testing. The sample below is common among organizations employing the Waterfall development model.

Requirements analysis: Testing should begin in the requirements phase of the software development life cycle. During the design phase, testers work with developers in determining what aspects of a design are testable and with what parameters those tests work.
Test planning: Test strategy, test plan, testbed creation. Since many activities will be carried out during testing, a plan is needed.
Test development: Test procedures, test scenarios, test cases, test datasets, test scripts to use in testing software.
Test execution: Testers execute the software based on the plans and test documents then report any errors found to the development team.
Test reporting: Once testing is completed, testers generate metrics and make final reports on their test effort and whether or not the software tested is ready for release.
Test result analysis: Or Defect Analysis, is done by the development team usually along with the client, in order to decide what defects should be assigned, fixed, rejected (i.e. found software working properly) or deferred to be dealt with later.
Defect Retesting: Once a defect has been dealt with by the development team, it is retested by the testing team.
Regression testing: It is common to have a small test program built of a subset of tests, for each integration of new, modified, or fixed software, in order to ensure that the latest delivery has not ruined anything, and that the software product as a whole is still working correctly.
Test Closure: Once the test meets the exit criteria, the activities such as capturing the key outputs, lessons learned, results, logs, documents related to the project are archived and used as a reference for future projects.

What Is The Difference Between Quality Assurance, Quality Control, And Testing?

Quality Assurance: A set of activities designed to ensure that the development and/or maintenance process is adequate to ensure a system will meet its objectives.

Quality Control: A set of activities designed to evaluate a developed work product.

Testing: The process of executing a system with the intent of finding defects. (Note that the "process of executing a system" includes test planning prior to the execution of the test cases.)

QA activities ensure that the process is defined and appropriate. Methodology and standards development are examples of QA activities. A QA review would focus on the process elements of a project - e.g., are requirements being defined at the proper level of detail. In contrast, QC activities focus on finding defects in specific deliverables - e.g., are the defined requirements the right requirements. Testing is one example of a QC activity, but there are others such as inspections. Both QA and QC activities are generally required for successful software development.

Quality control measures the quality of a product.
Quality assurance measures the quality of processes used to create a quality product.

Quality Assurance: Is oriented towards preventing defects. Quality Assurance ensures all parties concerned with the project adhere to the process and procedures, standards and templates and test readiness reviews.
Quality Control: quality control or quality engineering is a set of measures taken to ensure that defective products or services are not produced, and that the design meets performance requirements.

Quality Assurance:
1. Quality assurance meant for developing, organizing the best quality process
2. QA is process related
3. QA focuses on building in quality and hence preventing defects
4. QA: Deals with process
5. QA: for entire life cycle
6. Quality Assurance makes sure you are doing the right things, the right way.
7. QA is preventive process.

Quality Control:
1. Quality control meant for implementing the process developed by former team
2. QC is the actual testing of the software
3. QC focuses on testing for quality and hence detecting defects
4. QC: Deals with product
5. QC: for testing part in SDLC
6. Quality Control makes sure the results of what you've done are what you expected
7. QC is corrective process.

Quality Assurance:
- A planned and systematic set of activities necessary to provide adequate confidence that requirements are properly established and products or services conform to specified requirements.
- An activity that establishes and evaluates the processes to produce the products.

Quality Control:
- The process by which product quality is compared with applicable standards; and the action taken when nonconformance is detected.
- An activity which verifies if the product meets pre-defined standards.

Quality Assurance(QA): - It is an activity that is based on process where we measure each processes, identify any weakness and suggest improvement. The output of Quality Control (QC) activities often becomes the input to Quality Assurance (QA) Activities
2) Quality Control(QC):- It is an activity that is based on product where we measure the product, identify any weakness and improve the product. The change may include from single line code to reworking the whole product.

Difference Between QA & QC:

Quality Assurance (QA) Quality Control (QC)
QA is Process oriented, QC is Product oriented.
QA adopt Preventive approach, QC adopt Detective approach.
Audit, Review is an example of QA, Testing is a part of Quality Control.

Verification and Validation

Verification takes place before validation, and not vice versa.

Verification evaluates documents, plans, code, requirements, and specifications. Validation, on the other hand, evaluates the product itself.

The inputs of verification are checklists, issues lists, walkthroughs and inspection meetings, reviews and meetings. The input of validation, on the other hand, is the actual testing of an actual product.

The output of verification is a nearly perfect set of documents, plans, specifications, and requirements document. The output of validation, on the other hand, is a nearly perfect, actual product.

Validation vs. Verification
Verification and Validation (also known simply as V&V) are two parts of the same software package. They are used in software project management, software testing, and software engineering. It is the process by which a software system meets certain specifications. It is also the process by which a software system fulfills the intended purpose of its creation. It is also commonly known as software quality control.
Validation is the portion of the software checks and balances that checks that the product design satisfies or fits the use for which it was intended. This is known as high level checking (basically, informing the system that it built the right product). It carries out this task using dynamic testing and a variety of other forms of review. Dynamic testing specifically examines the physical response from the system to those variables that are not constant and, in time, are prone to change. In a basic sense, validation ensures that the product meets the needs of the user. It also ensures that the certain specifications were, in fact, correct from the beginning of the program. Basically, validation lets you know if you have built the right thing.
Verification is the portion of the software checks and balances that evaluates the software to determine whether the products that are found in a given development phase satisfy the conditions that were put forth at the beginning of that particular phase. In a basic sense, verification ensures that the particular product has been built according to the requirements and design specifications that were introduced at the beginning of the program. Quite frankly, verification lets you know that the correct object was built correctly.
Beyond the software community, the definitions of verification and validation are somewhat similar. In the modelling and simulation community, validation is the process by which the degree of accuracy of a model, simulation, or federation of models and simulations and their associated data can be determined. It also determines whether these models, simulations, or federations therein are accurate representations of the real world from the perspective of the use that was intended for the model, etc. Verification, on the other hand, is the process by which the system determines whether a computer model, simulation, or federation of models and simulations implementations and the content associated with that data represents the conceptual descriptions and specifications of the developer.
Summary:
1. Validation checks that a product design fits its intended use (if the right object has been built); verification evaluates the software that will determine whether the products found satisfy the conditions put forth (if the object built was done so correctly).
2. In the modelling and simulation community, validation determines the degree of accuracy of associated data to the real world according to the model; verification determines whether a computer model and the associated content represent the developer’s conceptual descriptions and specifications.

Read more: Difference Between Validation and Verification | Difference Between http://www.differencebetween.net/technology/difference-between-validation-and-verification/#ixzz0rZ9s3bXN

Servers

Server is a computing device, which is responsible for providing various services to its clients. The nature of these services can be anything, like file transferring, resource sharing, etc. Client and server systems provide functionalities such as centralized access to information, data storage and other resources.

Web Servers Explained

When you are trying to explore a web server, you have to consider what hyper text transfer protocol (HTTP) is and then proceed further. HTTP is an application level protocol, used for presenting hypermedia information system. Basically, this protocol is used when there are many different types of media involved. Well, web servers are those real or virtual machines that are responsible for delivering content. Clients are those machines or web browsers that request for any such content (HTML pages). Primarily, clients request for the content, but in some cases they have to deliver it as well (while submitting information for creating email accounts etc.).

For example: Consider that you wish to create an email account. First you type the web address of the email service provider in your browser. The terminal from which you are accessing the website is the client in this case. Once you enter the address, you will see that the URL is changed and now it is starting with "http://...". This action is the indication that hypertext transfer protocol is being used. In a matter of seconds, you see that web page of the email service provider are loaded onto your browser. This action indicates that the web server has provided content to the browser. And when you provide information to website (for opening an account), you are basically sending information to the web server.

Application Servers Explained

Application servers are basically those servers which support the dynamic content requests by different applications. They can be looked upon as a software framework, which do the job of efficient execution of procedures. Application servers are dedicated or designed for handling specific applications like database management systems like Oracle or MySQL. If the complexity involved in such an application is very large, then all the resources maybe dedicated to it. But this is a rare case, an application server is designed to handle many applications.

For example: Many a times, the information that client requests for, needs lot of computations. These computations maybe beyond the capabilities of the client itself. So they have to be done on the server and then the results are sent to client. An ATM machine is a real life example for this. An ATM does not credit or deposit any money in any bank account. This task is within the domain of the centralized servers of a particular bank. An ATM basically sends the request for withdrawal or deposit of the amount, these actions are then performed by the application server of the bank.

Client–server model

Client–server model of computing is a distributed application structure that partitions tasks or workloads between service providers, called servers, and service requesters, called clients. Often clients and servers communicate over a computer network on separate hardware, but both client and server may reside in the same system. A server machine is a host that is running one or more server programs which share its resources with clients. A client does not share any of its resources, but requests a server's content or service function. Clients therefore initiate communication sessions with servers which await (listen to) incoming requests.

Audio/Video Servers
Audio/Video servers bring multimedia capabilities to Web sites by enabling them to broadcast streaming multimedia content.

Chat Servers
Chat servers enable a large number of users to exchange information in an environment similar to Internet newsgroups that offer real-time discussion capabilities.

Fax Servers
A fax server is an ideal solution for organizations looking to reduce incoming and outgoing telephone resources but that need to fax actual documents.

FTP Servers
One of the oldest of the Internet services, File Transfer Protocol makes it possible to move one or more files securely between computers while providing file security and organization as well as transfer control.

Groupware Servers
A groupware server is software designed to enable users to collaborate, regardless of location, via the Internet or a corporate intranet and to work together in a virtual atmosphere.

IRC Servers
An option for those seeking real-time discussion capabilities, Internet Relay Chat consists of various separate networks (or "nets") of servers that allow users to connect to each other via an IRC network.

List Servers
List servers offer a way to better manage mailing lists, whether they be interactive discussions open to the public or one-way lists that deliver announcements, newsletters, or advertising.

Mail Servers
Almost as ubiquitous and crucial as Web servers, mail servers move and store mail over corporate networks (via LANs and WANs) and across the Internet.

News Servers
News servers act as a distribution and delivery source for the thousands of public news groups currently accessible over the USENET news network.

Proxy Servers
Proxy servers sit between a client program (typically a Web browser) and an external server (typically another server on the Web) to filter requests, improve performance, and share connections.

Telnet Servers
A Telnet server enables users to log on to a host computer and perform tasks as if they're working on the remote computer itself.

Verification

Verification refers to the collection of activities and procedures conducted during the planning and development, or after completion of an inventory that can help to establish its reliability for the intended applications of the inventory. For the purposes of this guidance, verification refers specifically to those methods that are external to the inventory and apply independent data, including comparisons with inventory estimates made by other bodies or through alternative methods. Verification activities may be constituents of both QA and QC, depending on the methods used and the stage at which independent information is used.

Verification ensures the product is designed to deliver all functionality to the customer; it typically involves reviews and meetings to evaluate documents plans code requirements and specifications; this can be done with checklists issues lists walkthroughs and inspection meetings. You can learn to do verification with little or no outside help.

Quality Assurance

Quality Assurance refers to the process used to create the deliverables, and can be performed by a manager, client, or even a third-party reviewer. Examples of quality assurance include process checklists and project audits. If your project gets audited, for instance, an auditor might not be able to tell if the content of a specific deliverable is acceptable (quality control). However, the auditor should be able to tell if the deliverable seems acceptable based on the process used to create it (quality assurance). That's why project auditors can perform a quality assurance review on your project, even if they do not know the specifics of what you are delivering. They don't know your project, but they know what good processes look like.

Here's an example to drive home the point. Let's say a project manager asked the sponsor to approve the Business Requirements Report. If you were the sponsor, how would you validate that the business requirements seemed complete and correct?
One solution would be for you to actually review the document and the business requirements. If you did that, you would be performing a quality control activity, since your actions would be based on validating the deliverable itself.

However, let's say the document was thirty pages long and that you (as the sponsor) did not have the expertise, the time, or the inclination to do a specific content review. In that case, you wouldn't ask to review the document itself. Instead, you would ask the project manager to describe the process used to create the document. Let's say you received the following reply.

Project manager - "I gathered eight of your major users in a facilitated session. After the meeting, I documented the requirements and asked the group for their feedback, modifications, etc. I then took these updated requirements to representatives from the Legal, Finance, Manufacturing and Purchasing groups and they added requirements that were needed to support company standards. We then had a meeting with the four managers in your area that are most impacted by this system. These managers added a few more requirements. I then asked your four managers to sign off on the requirements and you can see their signatures on the last page."

If you were the sponsor, would you now feel comfortable to sign the requirements? If it were me, I would feel pretty comfortable.

That's the difference. Quality control activities are focused on the deliverable itself. Quality assurance activities are focused on the process used to create the deliverable. They are both powerful techniques and both must be performed to ensure that the deliverables meet your customers quality requirements.

Quality Control

Quality Control refers to quality related activities associated with the creation of project deliverables. Quality control is used to verify that deliverables are of acceptable quality and that they are complete and correct. Examples of quality control activities include deliverable peer reviews and the testing process.

Performance Testing

Load and stress testing are subsets of performance testing.

Performance testing means how best something performs under a given benchmark. For example How much time you take to run 100 meters without carrying any load (no load is the benchmark)?
Load testing is also performance testing but under various loads. The previous example if extended would be How much time you took to run the same 100mts but carrying a load of 50 kilos, 100 kilos .... ?
Stress testing is performance under stress conditions. Extending the same example as before How much time you took to run 100 meters with load or no load when a strong wind was blowing in opposite direction ?.
Extending performance, load.. testing to s/w or h/w application.

For Example
Performance : 1000 txns per minute with 1000 users concurrent
Load : How many txns when 2000, 3000, 4000 concurrent users.
Stress : How many txns when 1000, 2000, 3000 concurrent users... under conditions like server memory very low, data transmission line poor, etc...

Web application

Web application is a bit different and complex to test as tester don’t have that much control over the application. Application is loaded on the server whose location may or may not be known and no exe is installed on the client machine, you have to test it on different web browsers. Web applications are supposed to be tested on different browsers and OS platforms so broadly Web application is tested mainly for browser compatibility and operating system compatibility, error handling, static pages, backend testing and load testing.

Client Server application

In client server application you have two different components to test. Application is loaded on server machine while the application exe on every client machine. You will test broadly in categories like, GUI on both sides, functionality, Load, client-server interaction, backend. This environment is mostly used in Intranet networks. You are aware of number of clients and servers and their locations in the test scenario.

Client–server model of computing is a distributed application structure that partitions tasks or workloads between service providers, called servers, and service requesters, called clients. Often clients and servers communicate over a computer network on separate hardware, but both client and server may reside in the same system. A server machine is a host that is running one or more server programs which share its resources with clients. A client does not share any of its resources, but requests a server's content or service function. Clients therefore initiate communication sessions with servers which await (listen to) incoming requests.

Software Testing Interview Questions and Answers

1. What is the purpose of the testing?

Software testing is the process used to help identify the Correctness, Completeness, Security and Quality of the developed Computer Software.

Software Testing
is the process of executing a program or system with the intent of finding errors.

2. What is quality assurance?

Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'.

3. What is the difference between QA and testing?

Testing involves operation of a system or application under controlled conditions and evaluating the results. It is oriented to 'detection'.

Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'.

4. Describe the Software Development Life Cycle

It includes aspects such as initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting, phase-out, and other aspects.

5. What are SDLC and STLC and the different phases of both?

SDLC
> Requirement phase
> Desing phase (HLD, DLD (Program spec))
> Coding
> Testing
> Release
> Maintenance

STLC
> System Study
> Test planning
> Writing Test case or scripts
> Review the test case
> Executing test case
> Bug tracking
> Report the defect

6. What is a Test bed?

Test Bed is an execution environment configured for software testing. It consists of specific hardware, network topology, Operating System, configuration of the product to be under test, system software and other applications. The Test Plan for a project should be developed from the test beds to be used.

7. What is a Test data?

Test Data is that run through a computer program to test the software. Test data can be used to test the compliance with effective controls in the software.

8. Why does software have bugs?

Miscommunication or no communication about the details of what an application should or shouldn't do

Programming errors in some cases the programmers can make mistakes.

Changing requirements there are chances of the end-user not understanding the effects of changes, or may understand and request them anyway to redesign, rescheduling of engineers, effects of other projects, work already completed may have to be redone or thrown out.

Time force - preparation of software projects is difficult at best, often requiring a lot of guesswork. When deadlines are given and the crisis comes, mistakes will be made.

9. What is the Difference between Bug, Error and Defect?

Error: It is the Deviation from actual and the expected value.

Bug: It is found in the development environment before the product is shipped to the respective customer.

Defect: It is found in the product itself after it is shipped to the respective customer.

10. Describe the difference between validation and verification

Verification is done by frequent evaluation and meetings to appraise the documents, policy, code, requirements, and specifications. This is done with the checklists, walkthroughs, and inspection meetings.

Validation is done during actual testing and it takes place after all the verifications are being done.

11. What is the difference between structural and functional testing?

Structural testing is a "white box" testing and it is based on the algorithm or code.

Functional testing is a "black box" (behavioral) testing where the tester verifies the functional specification.

12. Describe bottom-up and top-down approaches

Bottom-up approach: In this approach testing is conducted from sub module to main module, if the main module is not developed a temporary program called DRIVERS is used to simulate the main module.

Top-down approach: In this approach testing is conducted from main module to sub module. if the sub module is not developed a temporary program called STUB is used for simulate the submodule.

13. What is Re- test? What is Regression Testing?

Re- test - Retesting means we testing only the certain part of an application again and not considering how it will effect in the other part or in the whole application.

Regression Testing - Testing the application after a change in a module or part of the application for testing that is the code change will affect rest of the application.

14. Explain Load, Performance and Stress Testing with an Example.

Load Testing and Performance Testing are commonly said as positive testing where as Stress Testing is said to be as negative testing.

Say for example there is a application which can handle 25 simultaneous user logins at a time. In load testing we will test the application for 25 users and check how application is working in this stage, in performance testing we will concentrate on the time taken to perform the operation. Where as in stress testing we will test with more users than 25 with less memory and less network bandwidth and the test will continue to any number and we will check where the application is cracking.

15. What is UAT testing? When it is to be done?

UAT testing - UAT stands for 'User acceptance Testing. This testing is carried out with the user perspective and it is usually done before the release.


16. Functional testing (also known as black-box testing), is a software testing approach in which:
1. the tester will have a user perspective in mind,
2. not knowing and doesn't mind how the program works.
3. Input and output are the only things that matter.
4. The tester acts as if he/she is the final user of the program.

On the other hand, structural testing (also known as white-box testing), is a software testing approach in which:
1. the tester will have a developer perspective in mind,
2. knowing how the program works behind the scene,
3. such that the test will test all algorithm paths in the program.
4. Everything does matter.
5. The tester acts as a developer of the program who knows the internal structure of the program very well.

17. Difference between Client Server and Web Application?

In client server application you have two different components to test. Application is loaded on server machine while the application exe on every client machine. You will test broadly in categories like, GUI on both sides, functionality, Load, client-server interaction, backend. This environment is mostly used in Intranet networks. You are aware of number of clients and servers and their locations in the test scenario.

Web application is a bit different and complex to test as tester don’t have that much control over the application. Application is loaded on the server whose location may or may not be known and no exe is installed on the client machine, you have to test it on different web browsers. Web applications are supposed to be tested on different browsers and OS platforms so broadly Web application is tested mainly for browser compatibility and operating system compatibility, error handling, static pages, backend testing and load testing.

18. What is the role of the QA team on a software development project?

The QA Team is responsible for the quality of the Project by preventing and detecting bugs.
Help R&D team in meeting the schedules of the project.
Indicate the defect proven areas in the project.
Automation.
Analyze the Bugs Using Root cause Analysis.
To reduce overall cost and improve quality of the project.

19. What does 'regression testing' mean and why is it important to do it?

Testing conducted for the purpose of evaluating whether or not a change to the system has introduced a new failure.
Regression Testing is so important that the Product Stability is evaluated
When new functionality is added or existing functionality is modified, there is every possibility that new defects or side effects might have been  introduced.

20. What is automated testing? What makes automation hard to do ?

Automating the Test cases which are manually tested by a Tester.
Automation is hard for the following reasons.
1.Selection of proper automation tool that is suitable to the project.
2.Scripter (person responsible for automation) should  have awareness of the tool and basic programming skills.
3..Most of the projects' UI  controls does not fall under automation tool's traditional window classes. So need to find work arounds.
4.Automated scripts need to be documented properly and perfectly as scripts would be executed not by scripter alone but by entire QA team.
5.Controls or objects tags or UI frequently changes causing scripts to be changed frequently.

21. What are some likely differences between a QA and a Production environment? 

QA Environment is the Test Environment, the Project is tested on the Test Environment before moving on to the Production environment which will be fully operational.

QA environment is controlled whereas production environment is uncontrolled.

Tester knows how to use the product but  end user (in production environment) may or may not  be know about the product.

Any mistake in QA environment does not have impact on the company but same mistake when made in production environment brings bad name for the company.

 22. What is SQL and why might QA testers need to use it?

SQL is a Structured Query Language which is used for doing transactions like addition, deletion and modifications of records in the table and also for Querying the database.
As most of the products these days are based on databases Tester should  know about SQL.

23. Did you ever consider becoming a developer? Why did you choose QA for your career?

Initially when I was fresher (as soon as I was out of college)  I  thought of becoming a developer.
QA work interests me especially use of automation tools.
It is QA team which has more knowledge about a project than developers.
 

Different types of Automation Frameworks

The framework is made up of a number of reusable modules and function libraries that are developed with the following characteristics in mind:

Maintainable
Reusable
Manageable
Accessible
Robust
Flexibility
Measurable

Different Automation Frameworks are:

Ad-Hoc: Scripting developed in reactionary mode to test a single issue or fix.

Test script modularity framework requires the creation of small, independent scripts that represent modules, sections, and functions of the application-under-test. These small scripts are then used in a hierarchical fashion to construct larger tests, realizing a particular test case.

Data-Driven
Scripts are an assembly of function calls
Data for test cases read in from external source (e.g., Excel spreadsheet, ODBC database)
Results can be captured externally per script execution (i.e., spreadsheet, database)

Keyword-Driven
Test cases are expressed as sequence of keyword-prompted actions
A Driver script executes Action scripts which call functions as prompted by keywords
No scripting knowledge necessary for developing and maintaining test cases

Hybrid Automation Framework is a methodology uses a combination of three different technology approaches traditionally used Test Automation Framework development:

1. Modularity-driven testing
2. Data-Driven Testing
3. Keyword-driven testing

Black Box Tesing

Focus on functional requirements.

Attempts to find:

1. incorrect or missing functions
2. interface errors
3. errors in data structures or external database access
4. performance errors
5. initialization and termination errors.

Equivalence Partitioning

Divide the input domain into classes of data for which test cases can be generated.

Attempting to uncover classes of errors.

Based on equivalence classes for input conditions. An equivalence class represents a set of valid or invalid states

An input condition is either a specific numeric value, range of values, a set of related values, or a Boolean condition.

Equivalence classes can be defined by:

• If an input condition specifies a range or a specific value, one valid and two invalid equivalence classes defined.
• If an input condition specifies a Boolean or a member of a set, one valid and one invalid equivalence classes defined.

Test cases for each input domain data item developed and executed.

Boundary Value Analysis

Large number of errors tend to occur at boundaries of the input domain.

BVA leads to selection of test cases that exercise boundary values.

BVA complements equivalence partitioning. Rather than select any element in an equivalence class, select those at the ''edge' of the class.

Examples:

1. For a range of values bounded by a and b, test (a-1), a, (a+1), (b-1), b, (b+1).
2. If input conditions specify a number of values n, test with (n-1), n and (n+1) input values.
3. Apply 1 and 2 to output conditions (e.g., generate table of minimum and maximum size).
4. If internal program data structures have boundaries (e.g., buffer size, table limits), use input data to exercise structures on boundaries.

Cause Effect Graphing Techniques

Cause-effect graphing attempts to provide a concise representation of logical combinations and corresponding actions.

1. Causes (input conditions) and effects (actions) are listed for a module and an identifier is assigned to each.
2. A cause-effect graph developed.
3. Graph converted to a decision table.
4. Decision table rules are converted to test cases.

Comparison Testing

In some applications the reliability is critical.

Redundant hardware and software may be used.

For redundant s/w, use separate teams to develop independent versions of the software.

Test each version with same test data to ensure all provide identical output.

Run all versions in parallel with a real-time comparison of results.

Even if will only run one version in final system, for some critical applications can develop independent versions and use comparison testing or back-to-back testing.

When outputs of versions differ, each is investigated to determine if there is a defect.

Method does not catch errors in the specification.