Test Types
Functional, non-functional, structural, change-related. Learn the four categories of test types, their sub-types, and how to choose the right testing focus for the risk at hand.
1 The Hook
In 2020, a popular NZ online retailer launched a new checkout flow just before Black Friday. The QA team had run the full functional regression: add to cart, apply promo code, enter shipping address, pay with credit card, receive confirmation email. Every test passed. The team was confident.
Then the traffic hit. Within two hours of the sale starting, the site slowed to a crawl. Pages took 30 seconds to load. Customers in Auckland and Christchurch were abandoning carts in frustration. By midday, the payment gateway started timing out, and orders were being lost.
The functional tests were correct: the checkout flow worked. But nobody had tested whether it worked under load. Nobody had checked performance. Nobody had verified what happened when a thousand Kiwis tried to buy the same deal at 9 a.m. The functional tests had given a false sense of security because a critical non-functional test type was skipped.
This is why test types matter. Functional correctness is necessary but not sufficient. A system must also be fast, secure, usable, and reliable.
2 The Rule
ISTQB groups testing into four test types. Each type answers a different question about the software.
Functional Testing
Tests what the system does. Does it meet the specified functional requirements?
- Functional suitability: Completeness, correctness, appropriateness.
- Interoperability: Ability to interact with other systems.
- Security (functional aspect): Authentication, authorisation, access control.
Non-functional Testing
Tests how the system performs. Quality attributes and operational characteristics.
- Performance: Response time, throughput, resource usage.
- Load & stress: Behaviour under expected and peak load.
- Usability: Ease of use, learnability, accessibility.
- Reliability: Availability, fault tolerance, recoverability.
- Security (non-functional aspect): Confidentiality, integrity, resistance to attacks.
- Compatibility: Works across browsers, devices, operating systems.
- Portability: Ease of installation, adaptability, replaceability.
Structural (White-box) Testing
Tests how the code is built. Based on the internal structure of the software.
- Statement coverage: Has every line of code been executed?
- Decision coverage: Has every branch (true/false) been taken?
- Data flow testing: Are variables defined and used correctly?
Change-related Testing
Tests what changed and whether the change broke anything.
- Confirmation testing (re-testing): Re-run tests that failed to verify a fix.
- Regression testing: Re-run tests that passed to ensure nothing else broke.
- Maintenance testing: Testing after environment changes, data migrations, or upgrades.
3 The Analogy
Imagine you are opening a new restaurant on Cuba Street in Wellington.
Functional testing is checking the menu. Can a customer order a flat white? Does the kitchen receive the order? Is the correct dish served? Does the EFTPOS machine accept payment? This is what the restaurant does.
Non-functional testing is everything else. Is the coffee hot enough? Do customers get their meals within 15 minutes? Is there wheelchair access? Does the kitchen handle a full house on Friday night? Can the POS system stay online during a power fluctuation? This is how well the restaurant operates.
Structural testing is inspecting the kitchen itself. Are the fridges at the right temperature? Is the grease trap cleaned on schedule? Are staff washing hands between tasks? You are not checking a customer experience; you are checking the internal systems that make the experience possible.
Change-related testing is what you do after hiring a new chef. You re-test their signature dish to confirm it meets your standards (confirmation testing). You also check that the existing menu still tastes right, because the new chef might have reorganised the spice rack and subtly changed every dish (regression testing).
A restaurant can have a perfect menu and still fail if the service is slow, the kitchen is unhygienic, or a new hire breaks a winning formula. Software is the same.
4 Watch Me Do It
Let’s categorise tests for an NZ e-commerce site selling outdoor gear: tramping boots, tents, and fishing rods, shipping nationwide from a warehouse in Christchurch.
| Test | Type | Sub-type | Why It Matters |
|---|---|---|---|
| A customer can add a tent to cart, enter a rural NZ address, and pay with POLi. | Functional | Functional suitability | If the core purchase flow fails, the business makes no money. |
| The checkout completes within 3 seconds on 4G. | Non-functional | Performance | Kiwis in rural Otago shop on mobile. Slow checkout = abandoned cart. |
| The site is usable with a screen reader. | Non-functional | Usability / accessibility | Legal and ethical obligation under NZ accessibility standards. |
| All code paths in the shipping calculator are executed by tests. | Structural | Statement coverage | The rural surcharge logic might have a branch for “non-urban delivery” that is never hit in functional tests. |
| After fixing a bug in the discount engine, all previously passing checkout tests are re-run. | Change-related | Regression testing | A fix in one place can break another. The discount engine touches cart totals, tax, and shipping. |
| The site displays correctly on Safari (iPhone), Chrome (Android), and Edge (Windows). | Non-functional | Compatibility | NZ has high iPhone market share and many corporate users on Windows. |
5 When / When-not
| When to Focus on Each Type | When It Is Less Critical |
|---|---|
| Functional: Always. If the system does not do what it should, nothing else matters. | When the system is a prototype or proof-of-concept. You might deliberately skip non-functional concerns to validate an idea quickly. |
| Non-functional: When user experience, safety, or compliance is at stake. E-commerce needs performance. Healthcare needs reliability. Public sites need accessibility. | When building an internal tool with five users. If the admin dashboard takes 5 seconds to load and only the finance team uses it, that might be acceptable. |
| Structural: When code complexity is high, when safety standards demand coverage, or when you need to identify untested logic. | When the code is trivial (a single CRUD form) or when time pressure means coverage must be traded for functional confidence. |
| Change-related: After every bug fix, refactor, dependency update, or environment change. Regression suites are your safety net. | In a greenfield project with no existing behaviour to protect. Early sprints may focus on new functionality rather than regression. |
Before you decide which test types to focus on, ask:
- Do you truly understand the difference between a test type and a test level?
- Which non-functional properties matter most to your users: speed, security, accessibility, reliability?
- Are you testing types in isolation, or do you have regression tests for all types?
6 Common Mistakes
✗ Calling performance testing “functional”
Correction: Performance is non-functional. A payment form that works correctly but takes 40 seconds to submit is functionally correct and operationally unusable. Keep these categories separate in test plans and status reports so stakeholders understand that “all functional tests passed” does not mean “the system is ready for customers.”
✗ Confusing regression with re-testing
Correction: Re-testing (confirmation testing) verifies that a specific bug fix worked. Regression testing verifies that the fix did not break anything else. They are different activities with different scopes. Re-testing is surgical. Regression is broad. Do not report them as the same thing.
✗ Forgetting about compatibility testing
Correction: NZ has a diverse device landscape: iPhones, Samsung Galaxies, older Android handsets, iPads, desktops on Windows and macOS. Testing only on Chrome on a developer’s MacBook misses Safari flexbox bugs, Android keyboard issues, and Windows font rendering problems. Compatibility testing is non-functional and often overlooked.
When test type confusion fails
The biggest failure is confusing test types with test levels, or treating non-functional testing as a luxury that can be skipped. A team that focuses only on functional testing will ship broken user experiences, fail compliance audits, and miss critical security holes. Each test type finds different bugs. Skipping a type blinds you to an entire class of defects.
7 Now You Try
For each test case below, identify the test type and sub-type. Click to reveal.
Test Case A: After a developer fixes a crash in the “search by postcode” feature, the tester re-runs the failing search test to confirm the crash no longer occurs.
Change-related — Confirmation testing (re-testing). The tester is verifying that a specific fix resolved a specific failure. The scope is narrow and targeted.
Test Case B: A security firm attempts SQL injection, XSS, and brute-force attacks against a login page.
Non-functional — Security testing. The question is not “can a user log in?” but “can an attacker break in?” This tests resistance to attack, which is a quality attribute, not a business function.
Test Case C: A tester verifies that every “if” statement in the shipping module has been executed at least once by the test suite.
Structural — Decision coverage. This is about the internal code structure, not the external behaviour. The tester is checking whether the tests exercise all branches.
Test Case D: A tester checks that the GST amount on an invoice is calculated correctly at 15% for a $100 item.
Functional — Functional suitability / correctness. This verifies that the system performs a specified business calculation accurately.
8 Self-Check
Q1: A system passes all functional tests but crashes when 500 users log in simultaneously. Which test type was neglected?
Non-functional testing — specifically performance and load testing. The system works correctly for one user but fails under concurrent load. Functional correctness does not imply operational robustness.
Q2: What is the difference between re-testing and regression testing?
Re-testing (confirmation testing) verifies that a specific fix resolved a specific defect. It is narrow and targeted. Regression testing verifies that existing functionality still works after a change. It is broad and aims to detect unintended side effects.
Q3: Why is accessibility testing classified as non-functional rather than functional?
Accessibility is a quality attribute — it describes how the system is experienced by users with disabilities, not what the system does. A screen-reader user can still complete a purchase (functional), but if the experience is confusing or incomplete, the system fails on usability and accessibility (non-functional).
9 Interview Prep — Junior Q&A
Junior testers are often asked to articulate the difference between test types and levels. Be ready with clear examples.
Q. "What's the difference between a test type and a test level, and why does it matter?"
A test level is where you test (unit, integration, system, acceptance). A test type is what you test (functional, non-functional, structural, change-related). You need both dimensions. For example, you can do functional testing at every level, or do performance testing at the system and acceptance levels. If you confuse them, your test plans will be unclear and you'll skip important testing dimensions.
Q. "Give me an example of non-functional testing and why it matters."
Performance testing is non-functional. An e-commerce checkout that works correctly but takes 30 seconds to load will lose customers, even though it's functionally correct. Another example: a banking app that works fine on Chrome desktop but crashes on iOS. That's compatibility (non-functional) testing. Both find bugs that functional testing misses.
Q. "How would you explain the four test types to someone who's never heard of them?"
Functional tests ask, "Does it do what it's supposed to do?" Non-functional tests ask, "Does it do it fast enough, securely, reliably?" Structural tests ask, "Did the code get fully exercised?" Change-related tests ask, "Did the fix break anything else?" You need all four to have confidence in the system.