What is Quality?
· Customer
satisfaction? Subjective term. It will depend on who the ‘customer’ is. Each
type of customer will have their own view on ‘quality’
What is Software
Quality?
· Measurement
of how close is actual software product to the expected (intended) product
· Customer
satisfaction (to who?)
·
Quality Software: reasonably bug-free, delivered on time and within budget,
meets requirements and/or expectations, and is maintainable
What is Software
Quality Assurance?
·
Software QA is the process of monitoring and improving all activities
associated with software development, from requirements gathering, design and
reviews to coding, testing and implementation.
What is the
difference between Software Testing and Software QA?
· Testing
is mainly an ‘error detection’ process
· Software
QA is ‘preventative’. It aims to ensure quality in the methods & processes.
(“Quality Assurance” measures the quality of processes used
to create a quality product)
What is Software
Testing?
·
Software Testing is the process of analyzing the software in order to detect
the differences between existing and required conditions and to evaluate the
features of the software. It involves the
entire software development process:
- monitoring and improving the process
- making sure that any agreed-upon standards and procedures are followed
- ensuring that problems are found and dealt with, at the earliest possible
stage
· The
purpose of testing is verification, validation and error detection (in order to
find and fix the problems)
- Verification is checking for conformance and consistency by evaluating the results against pre-specified requirements. (Verification: Are we building the system right?)
- Validation is the process of checking that what has been specified is what the user actually wanted. (Validation: Are we building the right system?)
- Error Detection: finding if things happen when they shouldn’t or things don’t happen when they should.
- Verification is checking for conformance and consistency by evaluating the results against pre-specified requirements. (Verification: Are we building the system right?)
- Validation is the process of checking that what has been specified is what the user actually wanted. (Validation: Are we building the right system?)
- Error Detection: finding if things happen when they shouldn’t or things don’t happen when they should.
Is it possible to
find/fix all the bugs in a software product before it goes to the customers? Why
test?
· To
establish and to enforce business systems of the QA Organization (Test
planning, bug tracking, bug reporting, test automation, release certification,
and others)
What is
black/white box testing?
· Black
box software testing is done without access to the source code.
· White
box testing is done with access to the code. Bugs are reported at the source
code level, not behavioral.
Describe a bug?
· Mismatch
between actual behavior of a software application and its intended (expected)
behavior. We learn about expected behavior from requirements,
specifications, other technical documentation.
What is use case?
· Use
cases are used by Business Analysts as a format for specifying system
requirements. Each use case represents completed business operation performed
by user. From the QA prospective we would need to execute End-To-End test
to make sure the requirement is implemented.
· Find
more here: http://searchsoftwarequality.techtarget.com/sDefinition/0,,sid92_gci334062,00.html
What is the most
important impact QA can have on a product development process?
· Clarifying
requirements
· Bringing
down percentage of code re-written due to the change in requirements
What is Negative
testing? Positive?
· Positive
testing aimed at showing software works as intended when user does what
he/she does correct actions.
· Negative
testing aimed at showing that software handles properly situations in which
user acts not as user is supposed to act (invalid inputs, unreasonable
selections of settings, etc.)
Which type of
testing results in highest number of bugs found?
· Negative
testing (versus Positive testing of same type)
What is the
software development life cycle?
· The
software development life cycle (SDLC) is a conceptual model used in project
management that describes the stages involved in an information system
development project, from an initial feasibility study through maintenance of
the completed application. Read more -http://searchsoftwarequality.techtarget.com/sDefinition/0,,sid92_gci755068,00.html
What is a Test
Case?
- Set of conditions and/or variables
under which a tester will determine if a requirement upon an application
is satisfied
What does Test
Case include?
When planning for
testing the test case:
- Test case ID
- The purpose (Title, Description) of
the test case
- An instruction on how to get from
the application base state to a verifiable application
output or expected result
- Expected result
When execute test
cases we need two more columns:
- Actual result
- PASS/FAIL indication
What is a test
plan?
- Document
that describes the objectives, scope, approach, and focus of a software
testing effort.
- The process of preparing a test
plan is a useful way to think through the efforts needed to validate the
acceptability of a software product. The completed document will help
people outside the test group understand the 'why' and 'how' of product
validation. It should be thorough enough to be useful but not so thorough
that no one outside the test group will read it.
What does Test
Plan include?
The following are some of the items
that might be included in a test plan, depending on the particular project:
* Title
*
Identification of software including version/release numbers
* Revision
history of document including authors, dates, approvals
* Table of
Contents
* Purpose
of document, intended audience
* Objective
of testing effort
*
Software product overview
* Relevant
related document list, such as requirements, design documents, other test
plans, etc.
* Relevant
standards or legal requirements
*
Traceability requirements
* Relevant
naming conventions and identifier conventions
* Overall
software project organization and personnel/contact-info/responsibilities
* Test
organization and personnel/contact-info/responsibilities
*
Assumptions and dependencies
* Project
risk analysis
*
Testing priorities and focus
*
Scope and limitations of testing
* Test
outline - a decomposition of the test approach by test type, feature,
functionality, process, system, module, etc. as applicable
* Outline
of data input equivalence classes, boundary value analysis, error classes
* Test
environment - hardware, operating systems, other required software, data
configurations, interfaces to other systems
* Test
environment validity analysis - differences between the test and production
systems and their impact on test validity.
* Test
environment setup and configuration issues
* Software
migration processes
* Software
CM processes
* Test data
setup requirements
* Database
setup requirements
* Outline
of system-logging/error-logging/other capabilities, and tools such as screen
capture software, that will be used to help describe and report bugs
*
Discussion of any specialized software or hardware tools that will be used by
testers to help track the cause or source of bugs
* Test
automation - justification and overview
*
Test tools to be used, including versions, patches, etc.
* Test
script/test code maintenance processes and version control
* Problem
tracking and resolution - tools and processes
* Project
test metrics to be used
* Reporting
requirements and testing deliverables
* Software
entrance and exit criteria
* Initial
sanity testing period and criteria
* Test
suspension and restart criteria
*
Personnel allocation
* Personnel
pre-training needs
* Test
site/location
* Outside
test organizations to be utilized and their purpose, responsibilities,
deliverables, contact persons, and coordination issues
* Relevant
proprietary, classified, security, and licensing issues
* Open
issues
* Appendix
- glossary, acronyms, etc.
Write test cases
for a text field?
- 5 test cases for capacity including
2 for each boundary and one for the class between boundaries
- 3 test cases for valid/invalid
input of letters, digits, special characters
- One test cases for each allowed
special character (email field as an example)
- Functionality testing if there is
any functionality (validation of input as an example, case sensitivity,
required field, etc.)
What is Test matrix
Data collection mechanism. It
provides a structure for testing the effect of combining two or more variables,
circumstances, types of hardware, or events. Row and column headings identify
the test conditions. Cells keep the results of test execution.
If there are so
many settings/options to choose, how to write test cases?
- Test cases should be developed for
all most common potential scenarios
- They should cover most of the
positive input
Beside test case
& test plan, what documents are required to write?
· Check
Lists
· Test
matrices
· Test
design specs
· End-to-end
tests
· Test
summary reports
· Bug
reports
Describe risk
analysis
Risk analysis means the actions
taken to avoid things going wrong on a software development project, things
that might negatively impact the scope, quality, timeliness, or cost of a
project. This is, of course, a shared responsibility among everyone involved in
a project. However, there needs to be a 'buck stops here' person who can
consider the relevant tradeoffs when decisions are required, and who can ensure
that everyone is handling their risk management responsibilities.
How will you write
test cases for testing fields LOGIN & PASSOWRD, positive and negative
testing?
Testing boundary
conditions? Why? How?
- Boundary value analysis is a
methodology for designing test cases that concentrates software testing
effort on cases near the limits of valid ranges.
- Boundary value analysis is a method
which refines equivalence partitioning. It generates test cases
that highlight errors better than equivalence partitioning. The trick is
to concentrate software testing efforts at the extreme ends of the equivalence
classes. At those points when input values change from valid to
invalid errors are most likely to occur. As well, boundary
value analysis broadens the portions of the business requirement document
used to generate tests.
For example, if a valid range of quantity on hand is -9,999 through 9,999, write test cases that include:
1. the valid test case quantity on hand is -9,999,
2. the valid test case quantity on hand is 9,999,
3. the invalid test case quantity on hand is -10,000 and
4. the invalid test case quantity on hand is 10,000
What is the
difference between a test case and a test plan?
- Test plan is the most comprehensive
Software Testing document that describes the objectives, scope,
approach, and focus of a software testing effort
- Test case is the smallest Software
Testing document that describes both typical and atypical situation (set
of conditions and/or variables) that may occur in the use of
an application (under which a tester will determine if a requirement upon
an application is satisfied).
Which documents
would you refer to when creating Test Cases?
All business and technical
documentation available:
- BRD - Business Requirements
Document
- Functional Specifications
- Manuals and Help
- Use Cases
- Test Design
- Third party publications (books,
published by independent authors)
What is Business
Requirements Document (BRD)?
BRD is written by the Business
Analysts. It details the business
solution for a project including the documentation of customer needs and
expectations.
The
most common objectives of the BRD are:
-
To gain agreement with stakeholders
-
To provide a foundation to communicate to a technology service provider what
the solution needs to do to satisfy the customer's and business’ needs
-
To provide input into the next phase for this project
-
To describe what not how the customer/business needs will be met by the
solution
What are Bug
Report components?
What fields do you
fill out in a Bug Report?
Describe to me the
basic elements you put in a defect/bug report?
· Report
number: Unique number given to the report
· Application
/ Module being tested
· Version
& release number
· Problem
Summary / Short Description / Synopsis
· Steps
to reproduce (Detailed Description)
· Severity
(Critical, Serious, Minor, Suggestion)
· Priority
(High, Medium, Low)
· Environment
(Software and/or hardware configuration)
· Reported
by
· Assigned
to
· Status
(Open, Pending, Fixed, Closed, cannot reproduce, etc.)
· Resolution
/ Notes
· Keywords
If you find a bug
and the developer says it is as-designed, what can you do?
- find an exact requirement, which
defines the way it should be designed
- if there is no specific requirement compare to same feature implemented in quality applications (ask your manager which applications to compare to)
- if there is no specific requirement compare to same feature implemented in quality applications (ask your manager which applications to compare to)
How do you write a
bug report?
· Rule of WWW -
What happened, Where it happened, under Which circumstances
· Write
one bug report for each fix to be verified
· Bug
report should be as complete as possible
· Bug
reports are as concise as possible
· Report
a bug immediately, do not postpone
· Use
technical terms, not "people off the street" language
What is the most
important part of bug report?
· Steps
to reproduce
· Short
Description
· Severity
· Priority
· Status
What is the bug
life cycle?
The bug should go through the life
cycle to be closed. Here are the stages:
- bug found
- bug reported
- bug assigned to developer
- bug fixed by developer
- fix verified by tester
- bug closed
- bug found
- bug reported
- bug assigned to developer
- bug fixed by developer
- fix verified by tester
- bug closed
How can a tester
be sure that bug was fixed?
- execute the steps in
the bug report
- make sure the fixed bug does not result in new bugs in same area.
- make sure the fixed bug does not result in new bugs in same area.
Describe the QA
Process
QA processes include:
1) Test Planning Process
2) Test Development Process
3) Test Execution Process
4) Defect Management Process
5) Test Reporting Process
1) Test Planning Process
2) Test Development Process
3) Test Execution Process
4) Defect Management Process
5) Test Reporting Process
What is Unit
Testing?
· The
goal of unit testing is to isolate each part of the program and show that the
individual parts (units) are correct.
· A
unit is the smallest testable part of an application. It may be an individual
function or procedure.
· Unit
testing is provided by developers, not testers.
What is
· Testing
of an API (Application Programming
Interface), which is a collection of software functions and procedures.
· API testing
is mostly used for testing system software, application software or libraries.
· It
is a white box testing method.
· API testing
(done by QA Team) is different from Unit testing (done by developers).
What is the
Performance Testing? ?
Performance testing is to determine
how fast some aspect of a system performs under a particular workload. It can
also serve to validate and verify other quality attributes of the system, such
as scalability, reliability and resource usage.
What is Stress
Testing?
Stress test
puts a emphasis on robustness, availability, and error handling under
a heavy load, rather than on what would be considered correct behavior under
normal circumstances. The goal may be to ensure the software doesn't crash in
conditions of insufficient computational resources (such as memory or disk
space), unusually high concurrency, or denial of service attacks.
What is a
Regression Testing?
Partial retesting of a modified
program to make sure that no errors were introduced while making changes to the
code (developing new or fixing existing one)
What is an Acceptance Testing?
Acceptance testing is black-box
testing performed on a software prior to its delivery. Acceptance testing by
the system provider is distinguished from acceptance testing by the customer
(user acceptance testing - UAT).
What do you
prefer: white or black box testing?
- Stick to the objective stated in
your resume (Portnov School
How do you
determine when you have done enough testing?
Testing process comes to the point
at which additional tests will not significantly change quality of the software.
Which tools are
used to write Test Cases?
- Test Management Tools such as HPQuality Center
- Test Management Tools such as HP
- Many companies use spreadsheets
(Excel) or word processors (Word)
What is
walk-through meeting?
Walk-through meeting is a form of software peer review in which a designer or programmer leads members of the development team and other interested parties through a software product, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems.
Walk-through meeting is a form of software peer review in which a designer or programmer leads members of the development team and other interested parties through a software product, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems.
What is Build?
In a programming context, a build is a version of a program. As a rule, a build is a pre-release version and as such is identified by a build number, rather than by a release number. Reiterative (repeated) builds are an important part of the development process. Throughout development, application components are collected and repeatedly compiled for testing purposes.
What is Test
Strategy?
A test strategy is an outline that
describes the testing portion of the software development cycle. It is created
to inform project managers, testers, and developers about some key issues of
the testing process.
What does Test
Strategy include?
This includes the testing objective, methods of testing new functions, total time and resources required for the project, and the testing environment.
This includes the testing objective, methods of testing new functions, total time and resources required for the project, and the testing environment.
