Specification-based Testing

software-engineering testing

Definition

Specification-based Testing

Specification-based testing is a test design technique where test cases are derived directly from the requirements specification or functional design, without reference to the internal code structure. It is the primary method used in black-box testing.

Core Techniques

Equivalence Partitioning

Definition

Equivalence Partitioning

Equivalence Partitioning is a specification-based testing technique that divides the input data of a software unit into partitions of equivalent data from which test cases can be derived.
The fundamental premise is that if one test case in a partition detects a defect, other cases in the same partition would likely detect the same defect.

Link to original

Boundary Value Analysis

Definition

Boundary Testing

Boundary Testing is a specification-based testing technique that tests inputs at the boundaries of input domains. It is based on the observation that defects are more likely to occur at the edges of equivalence partitions than within the “centre”.

Link to original

Decision Table Testing

Decision Table Testing

A technique used to capture complex logic relationships between inputs (conditions) and outputs (actions). It is particularly useful when system behaviour depends on a combination of inputs.

Process

The systematic 7-step approach to identify partitions and derive tests is as follows:

1. Understand the Requirements: Analyse the specification to identify inputs, outputs, and rules.
2. Explore the Program: Experiment with the program (if available) using “happy path” inputs to understand basic functionality.
3. Identify Input Partitions:

• Look at Inputs: Explore types and ranges of each variable individually.
• Look at Interactions: Identify dependencies or constraints between variables.
• Look at Outputs: Identify all possible output types and work backwards.

1. Analyse Boundaries: Identify boundaries between partitions (on-points and off-points).
2. Devise Test Cases: Create concrete test cases for the identified partitions and boundaries.
3. Automate Tests: Implement the test cases in code.
4. Augment the Test Suite: Use experience and intuition to add cases that systematic partitioning might miss (e.g., duplicates, special characters).

Example

Example: alternateMerge

Specification: Combines two integer arrays by interleaving their elements. If one array runs out, the remaining elements of the other are appended. If either array is null, it is treated as empty. Returns a new, non-null array.

1. Input Partitions We identify individual variable states and their interactions.

• Individual Variables:
  • $P_{null}$: Input is null.
  • $P_{empty}$: Input is [].
  • $P_{pop}$: Input is populated (length \ge 1).

• Interactions:
  • Validity: One is P_{null}/P_{empty}, other is P_{pop}.
  • Length: L_1 = L_2, L_1 > L_2, L_1 < L_2.

2. Boundary Analysis

• Boundary: Equality of lengths ($L_1=L_2$).
• On Point: $L_1=L_2$ (Perfect interleave).
• Off Point 1: $L_1=L_2+1$ (Append arr1 tail).
• Off Point 2: $L_2=L_1+1$ (Append arr2 tail).

3. Concrete Test Cases:

ID	Input arr1	Input arr2	Rationale	Expected Outcome
T1	null	null	Partition: P_{null} (Base exception handling).	[]
T2	[]	[]	Partition: P_{empty}.	[]
T3	[1, 2]	[3, 4]	Boundary (On Point): L_1 = L_2.	[1, 3, 2, 4]
T4	[1, 2, 3]	[4, 5]	Boundary (Off Point 1): L_1 > L_2.	[1, 4, 2, 5, 3]
T5	[1, 2]	[3, 4, 5]	Boundary (Off Point 2): L_1 < L_2.	[1, 3, 2, 4, 5]
T6	null	[1, 2]	Interaction: Verify arr1 is treated as empty.	[1, 2]
T7	[1, 2]	null	Interaction: Verify arr2 is treated as empty.	[1, 2]
T8	[0, 0]	[1, 1]	Augment: Check for accidental deduplication.	[0, 1, 0, 1]

Example: passwordStrength

Specification: Evaluates password strength. Raises IllegalArgumentException if null or empty.

• Length < 6: “Weak”
• Length >= 6: Check content (Digit AND Upper \to Strong, XOR \to Medium, Neither \to Weak).

1. Input Partitions

• Validity: $P_{null}$ (null), $P_{empty}$ ("").
• Length: $P_{short}$ (< 6), $P_{long}$ ($\ge 6$).
• Content: (Dependent on $P_{long}$)
  • No Digit/Upper
  • Upper Only
  • Digit Only
  • Both

2. Boundary Analysis

• Boundary: Length = 6 (Explicit numeric value).
• On Point: 6 (Smallest length where content matters).
• Off Point: 5 (Largest length where content is ignored).

3. Concrete Test Cases

ID	Input pwd	Rationale	Expected Outcome
T1	null	Partition: $P_{null}$.	IllegalArgumentException
T2	""	Partition: $P_{empty}$.	IllegalArgumentException
T3	"Ab1de"	Boundary (Off Point): Length 5.	"Weak"
T4	"abcdef"	Boundary (On Point): Length 6. No Upper/Digit.	"Weak"
T5	"Abcdef"	Boundary (On Point): Length 6. Upper Only.	"Medium"
T6	"1bcdef"	Boundary (On Point): Length 6. Digit Only.	"Medium"
T7	"Abcde1"	Boundary (On Point): Length 6. Both.	"Strong"
T8	"A 1 "	Augment: Invisible characters count toward length.	"Strong"

Example: shippingCost

Specification: Computes cost from weight and speed. Throws exception for invalid inputs.

• Weights: $(0,5]\to 5$, $(5,20]\to 10$, $(20,\infty )\to 20$.
• Speed: “standard” (\times 1), “express” (\times 1.5). Case-insensitive.

1. Input Partitions

• Weight:
  • $w\le 0$ (Invalid)
  • $0<w\le 5$
  • $5<w\le 20$
  • $w>20$

• Speed: Invalid, Valid (“standard” / “express”).

2. Boundary Analysis

• Validity ($w>0$): On=0 (Invalid), Off=1 (Valid).
• Tier 1 ($w\le 5$): On=5 (Base 5), Off=6 (Base 10).
• Tier 2 ($w\le 20$): On=20 (Base 10), Off=21 (Base 20).

3. Concrete Test Cases

ID	Weight	Speed	Rationale	Expected Outcome
T1	0	"standard"	Boundary: w=0 (Invalid).	IllegalArgumentException
T2	1	"standard"	Boundary: w=1 (Valid).	5.0
T3	5	"express"	Boundary: w=5.	7.5
T4	6	"standard"	Boundary: w=6.	10.0
T5	20	"EXPRESS"	Boundary: w=20 (Case-insensitive).	15.0
T6	21	"standard"	Boundary: w=21.	20.0
T7	10	"fast"	Interaction: Invalid Speed.	IllegalArgumentException
T8	10	null	Interaction: Null Speed.	IllegalArgumentException
T9	10	" standard "	Augment: Strict string matching (whitespace).	IllegalArgumentException