Continuous Integration
Developers integrate code into a shared repository frequently — usually multiple times per day — with automated build and test verification.
What it is
Continuous Integration (CI) is the practice of merging all developer working copies to a shared main branch several times a day. Each integration is verified by an automated build and automated tests, giving the team immediate feedback on whether the change broke anything. The goal is simple: the main branch should always be in a working state.
When a developer commits, the CI server checks out the code, compiles it, runs the test suite, and reports the result. If the build fails, the team stops and fixes it before doing anything else. This discipline prevents the integration hell that occurs when multiple long-lived branches are merged at the end of a sprint, each containing weeks of conflicting changes.
The CI mantra: Commit to main daily. If the build is red, fixing it is the highest priority work in the team.
When to use it
Every code commit, multiple times per day. CI is not a scheduled activity like a nightly build; it is a continuous, automatic response to every push. Teams practising CI should be integrating at least once per developer per day, and often far more frequently.
CI is foundational for teams that also practise Continuous Delivery or Deployment, because you cannot safely deploy code that has not been continuously integrated and tested.
Key benefits: faster bug detection, reduced merge conflicts, immediate feedback on quality, living documentation through test results, and a main branch that is always deployable.
Key concepts
The Integration Problem
When developers work on isolated branches for days or weeks, they diverge from each other and from main. The longer the isolation, the harder and riskier the merge. CI solves this by forcing small, frequent integrations so conflicts remain small and manageable.
The Build Pipeline
A CI pipeline is a sequence of automated stages: compile, unit test, integration test, static analysis, package. Each stage must pass before the next begins. Pipelines are defined as code (YAML in GitHub Actions, GitLab CI, Azure Pipelines, or Jenkinsfile) so they are version-controlled and repeatable.
Fix or Revert
If a commit breaks the build, the author has two choices: fix it immediately, or revert the commit to restore main to green. There is no third option of leaving it red while finishing another task. A red main is a blocked team.
Pre-commit Checks
Fast feedback is better when it happens before the push. Pre-commit hooks run linters, formatters, and fast unit tests locally so the developer catches issues before CI even sees them. They should be quick (under a minute) or developers will skip them.
| Stage | Purpose | Typical Duration |
|---|---|---|
| Compile / Build | Ensure code is syntactically valid and dependencies resolve | 1–3 minutes |
| Unit Tests | Verify individual components in isolation | 2–5 minutes |
| Integration Tests | Verify components work together with real or test doubles | 5–15 minutes |
| Static Analysis | Catch code smells, security issues, and style violations | 1–3 minutes |
Common pitfalls
- Long-running feature branches. A branch that lives for a week is not CI. If you must use branches, keep them under a day and merge to main constantly.
- Ignoring broken builds. A red build that sits for hours trains the team to ignore CI. Fix it or revert it immediately.
- Slow builds. If the pipeline takes 45 minutes, developers will batch commits and context-switch while waiting. Aim for under 10 minutes for the feedback that matters most.
- Insufficient test coverage. CI without meaningful tests is just automated compilation. You need tests that can actually catch regressions.
- Committing without local tests. Pushing code you have not run locally is rude to your team. Run the relevant tests before pushing, every time.
Red flag: If your team has a “code freeze” before release because main is unstable, your CI practice has broken down. The freeze is a symptom; the cure is smaller commits and stronger tests.
NZ context
Government digital teams in New Zealand, including those under the Digital Service Design Standard, require Continuous Integration as a baseline capability. Teams working with agencies such as CoverNZ, Revenue NZ, or Benefits NZ must demonstrate that their main branch is automatically built and tested on every commit.
Smaller NZ consultancies and startups often run CI on cloud-hosted runners (GitHub Actions, GitLab CI) rather than maintaining their own infrastructure. This keeps costs predictable and scales with team size. For teams handling sensitive data, self-hosted runners within New Zealand cloud regions meet data residency expectations while retaining automation benefits.
Career level guidance
| Level | Focus | Milestones |
|---|---|---|
| Junior | Commit to main daily; run tests locally before pushing; read build logs | Understands pipeline stages; can diagnose and fix their own build failures; writes tests that run in CI |
| Senior | Design pipeline architecture; optimise build times; enforce fix-or-revert discipline | Reduces build time by 50% or more; sets up parallel test execution; coaches team on CI hygiene |
| Test Lead | Define quality gates; measure pipeline health; integrate security and compliance checks | Tracks build pass rate and mean time to recovery; implements mandatory pre-commit checks; audits pipeline for coverage gaps |
Tip for test leads: Publish a dashboard showing build pass rate, average build duration, and mean time to recovery (MTTR) after a red build. What gets measured gets improved.
Senior engineer insight
Teams that do CI well treat a red build as a fire alarm, not a background notification — the author stops what they are doing and fixes or reverts within the hour, every time. The single pattern that separates high-performing NZ DevOps teams from the rest is trunk-based development with short-lived branches (under four hours), which removes the temptation to batch changes and keeps the feedback loop tight. When your pipeline is fast enough that a developer never waits more than eight minutes, CI becomes the natural rhythm rather than an imposition.
The most common mistake: teams install a CI tool, declare victory, and keep merging long-lived feature branches every two weeks — they now have expensive automation running on the wrong rhythm.
From the field
A Wellington-based fintech team migrated from Jenkins to GitHub Actions and assumed the migration alone would fix their CI discipline. They kept their existing pattern of two-week feature branches, and the new pipeline was green exactly twice in the first month — both times were hotfixes on main. After a retrospective, they introduced a team norm: no branch lives longer than one working day, with feature flags to hide incomplete work. Within six weeks their pipeline pass rate went from under 30% to over 90%, and the weekly "merge Thursday" ritual (and its attendant two-hour conflict sessions) disappeared entirely. The lesson: tooling changes without behavioural changes achieve nothing — CI is a discipline first and a pipeline second.
Industry Reality
- Most teams have CI in name only — a pipeline exists but the build is permanently red, ignored, or bypassed with
--no-verify. Senior engineers spend real energy just keeping the signal trustworthy. - Long-lived feature branches (two to four weeks) remain the norm in many NZ shops despite CI being "in place". The pipeline runs but integration still happens once at the end, so you get the worst of both worlds.
- Build times creep past 30 minutes as test suites grow and nobody owns pipeline performance. Teams work around it by merging less frequently, which defeats the whole purpose.
- Flaky tests are the silent killer of CI trust. A pipeline that fails randomly 20% of the time trains developers to re-run rather than investigate, masking real failures for hours or days.
- Security and compliance checks are bolted on late — often only after an audit or incident — rather than designed into the pipeline from day one. Government-adjacent NZ projects increasingly require SAST and dependency scanning in every pipeline run.
Context guide
How the right level of Continuous Integration effort changes based on team context.
| Context | Priority | Why |
|---|---|---|
| Government agency delivering regulated services (CoverNZ, Revenue NZ, Benefits NZ) | Essential | Regulatory audit trails require every deployment to be traceable to a passing pipeline. The Digital Service Design Standard mandates automated testing on every commit. A broken main branch in a benefits or tax system can block downstream releases that affect citizens. |
| Financial services team (Harbour Bank, Pacific Bank, KiwiSaver provider) | Essential | Reserve Bank and Financial Markets Authority oversight requires demonstrable change controls. CI pipelines with mandatory security scanning (SAST, dependency audit) satisfy change-advisory board evidence requirements and reduce manual gate review time. |
| High-frequency consumer product (Pacific Air booking, TeleNZ self-service portal) | Essential | Multiple teams commit to shared codebases daily. Without CI, merge conflicts and silent regressions reach customers before anyone notices. Fast pipeline feedback protects revenue-critical flows and supports multiple releases per day. |
| Small NZ agency or consultancy (3–6 developers, single product) | High | Even small teams benefit because integration pain grows non-linearly with branch age. A simple GitHub Actions pipeline with unit and smoke tests costs almost nothing to run and prevents the “merge Friday” bottleneck that derails small teams before demos. |
| Legacy monolith with no automated tests (pre-transformation state) | Medium | A CI pipeline with no meaningful tests provides false assurance. Start by adding a build step and static analysis, then invest in a characterisation test layer before treating the green light as a quality signal. Phase in CI alongside test coverage growth. |
| Solo developer maintaining an internal tool with infrequent changes | Low | Integration conflicts are impossible when there is only one developer. A lightweight pre-commit hook and manual test run before deploy may be proportionate. CI still adds value for the automated build artefact and audit record, but is not urgent. |
Trade-offs
What you gain and what you give up when you adopt Continuous Integration.
| Advantage | Disadvantage | Use instead when… |
|---|---|---|
| Regressions are caught within minutes of the commit that causes them, while the developer still holds full context of their change. | A slow or poorly scoped pipeline creates a false sense of quality — a 40-minute pipeline that developers bypass is worse than no pipeline at all. | Your test suite is so slow or flaky that CI is actively misleading. Fix or quarantine flaky tests before trusting the signal; otherwise a nightly extended suite may be more honest. |
| Merge conflicts stay small because integration windows are measured in hours, not weeks. Teams avoid the sprint-end merge marathon. | CI requires the whole team to adopt trunk-based or short-branch discipline. One developer who keeps a two-week branch undermines the benefits for everyone. | The team is geographically distributed across incompatible time zones and cannot co-ordinate a shared merge cadence. Async-friendly branching strategies may be a transitional step. |
| Pipeline as code (committed YAML) means build configuration is version-controlled, peer-reviewed, and rolls back with the code — no undocumented Jenkins magic. | Initial pipeline setup and ongoing maintenance carry real cost — runner infrastructure, secrets management, caching strategy, and test environment stability all require engineering time. | The project is a short-lived proof of concept or throwaway prototype. A basic pre-commit hook and manual smoke test before each demo may be proportionate until the codebase is worth investing in. |
| Security and compliance checks (SAST, dependency scanning, licence audits) run automatically on every commit — vulnerabilities are found before they reach production or an audit. | Each additional pipeline stage (security scan, performance baseline, accessibility check) adds duration. Teams must actively manage pipeline performance or the feedback loop degrades over time. | Your compliance framework requires a human sign-off at every integration point (some regulated NZ environments under NZISM controls). Automated checks still run, but they feed a manual approval step rather than auto-merging. |
Enterprise reality
How CI looks at 200–300-developer scale in NZ enterprise — banks, government, telcos
- Pacific Bank runs mandatory SAST and dependency vulnerability scanning in every pipeline run to satisfy Reserve Coastal Bank (RBNZ) BS11 operational resilience requirements — a single failing CVE above a configurable severity threshold blocks the merge automatically, replacing what was previously a weekly manual review gate.
- At this scale, pipeline governance is automated rather than social: branch protection rules, required reviewer counts, and status checks are enforced in GitHub Enterprise or Azure DevOps so no individual developer or team lead can bypass them under delivery pressure — the controls exist at the platform level, not in a team agreement.
- Coordinating CI across 10–20 squads on a shared codebase requires explicit pipeline ownership — large NZ telcos such as TeleNZ assign a dedicated platform engineering team to own runner infrastructure, caching strategy, and build time SLAs; without this, build times creep past 40 minutes within two years as each squad adds stages without anyone removing them.
- Privacy Act 2020 obligations and NZISM controls mean self-hosted runners inside NZ or Australian cloud regions are often non-negotiable for government-adjacent pipelines — a misconfigured shared runner that sends build artefacts to an overseas region has caused real remediation costs for NZ public sector projects, including audit findings and delayed go-lives.
◆ What I would do
Professional judgment — when to adopt Continuous Integration, when to adapt it, and what to watch for.
The bottom line: A CI pipeline with no behavioural change is expensive decoration — the discipline of integrating to main every single working day, and treating a red build as a fire alarm rather than background noise, is what separates teams that get the benefit from teams that just have the tooling.
Best Practices
- ✓ Keep the main branch green as a non-negotiable team norm — fix or revert within the hour, never leave it red overnight.
- ✓ Target a pipeline completion time under 10 minutes; parallelize test stages and cache dependencies aggressively to stay there as the suite grows.
- ✓ Define pipeline as code (GitHub Actions YAML, Jenkinsfile, etc.) committed alongside application code so changes are peer-reviewed and roll back with the code.
- ✓ Quarantine flaky tests immediately into a separate "quarantine" stage rather than deleting them; track them as first-class bugs and fix them within the sprint.
- ✓ Enforce a pre-commit hook for fast checks (lint, type-check, unit tests under 60 seconds) so the CI server is your second line of defence, not your first.
- ✓ Use branch protection rules to block merges from a red pipeline — don't rely on social pressure alone, make it technically impossible to skip CI.
- ✓ Surface build metrics (pass rate, MTTR, duration trend) on a team dashboard or in your sprint review so pipeline health is a visible quality indicator, not a background concern.
- ✓ When running on self-hosted runners for data residency (common in NZ government work), treat runner maintenance as infrastructure work and schedule it explicitly — stale runners silently break pipelines.
Common Misconceptions
❌ Myth: "We have CI — we run a nightly build."
Reality: A nightly build is a scheduled batch process, not Continuous Integration. CI means every commit triggers a build and test run automatically, providing feedback in minutes rather than hours. Running once per day means problems can pile up for a full working day before anyone knows.
❌ Myth: "CI is a DevOps/developer concern — testers just consume the results."
Reality: Testers own a critical slice of CI: the test suite that runs in the pipeline. A QA who understands pipeline design can shift test execution earlier, reduce build times by classifying tests correctly, and define the quality gates that determine when a build is green. Active involvement in CI is a core senior tester skill, not an optional extra.
❌ Myth: "More tests in CI always means better quality."
Reality: Slow, flaky, or duplicated tests in CI actively harm quality by making the signal unreliable and slowing feedback loops. Experienced teams curate their CI test suite: fast unit and integration tests run on every commit, slower end-to-end tests run on a schedule or pre-release. The right tests at the right stage matter more than total count.
Self-Check
Click each question to reveal the answer.
Q1: What is the core discipline that separates genuine CI from a team that just has a build server?
The discipline is integrating to main every day and treating a red build as the team's highest priority. A build server that runs once a night on a long-lived branch is a scheduled batch process, not CI. The defining characteristic is the commitment to keeping main green at all times, not the existence of automation.
Q2: Why does CI reduce merge conflicts rather than just detect them?
CI forces small, frequent integrations to a shared branch, so divergence between developer copies stays small. When everyone merges to main at least once a day, the window for conflicting changes is short. Long-lived branches are where merge conflicts accumulate; CI's discipline of integrating daily eliminates the conditions that allow them to grow.
Q3: What is the recommended maximum pipeline duration, and why does that threshold matter?
The widely accepted target is under 10 minutes for the feedback that matters most (compile, unit, integration). Beyond that, developers switch context while waiting, start batching commits, and integrate less frequently — which undermines the whole purpose of CI. Fast feedback is the mechanism; the 10-minute ceiling is the constraint that keeps the feedback useful.
Q4: What should happen when a CI build breaks, and what should NOT happen?
The author should either fix the build immediately or revert the commit to restore main to green — and this should happen within the hour. What should not happen: leaving the build red while continuing other work, pushing more commits on top of a broken build, or treating a red build as normal background noise. A red main blocks the entire team from getting reliable feedback on their own changes.
Q5: Why are flaky tests particularly damaging to CI, even when they occasionally pass?
A flaky test trains the team to re-run rather than investigate failures. Once developers learn they can get a green by retrying, real failures get masked — a genuine regression slips through because everyone assumes the failure is just noise. The pipeline loses its signal value, which is the only thing that makes CI useful. Flaky tests should be quarantined and treated as first-class bugs.
Q6: Your team is delivering a new online claims portal for CoverNZ. The product owner wants a two-week feature branch for the entire claims submission flow before merging. You are the QA lead. How do you respond, and what approach would you propose instead?
A two-week branch defeats CI: changes accumulate, conflicts compound, and the team gets no integration feedback until the end when it is most expensive to fix. I would propose breaking the feature into trunk-compatible slices using a feature flag — the claims form can ship hidden behind a flag, integrated and tested continuously, and toggled on only when complete. Each slice merges to main daily, the pipeline stays green, and the team gets early feedback on the integration points (authentication, data validation, document upload) before they become expensive surprises at the end of the sprint. For CoverNZ's compliance requirements, this also means security and data residency checks run in the pipeline from day one rather than being discovered in a late-stage review.
Q7: An Revenue NZ system currently runs a nightly build that emails the team a pass/fail report each morning. The delivery manager says "we already have CI — the build runs every night." What would you say, and what changes would you propose to move the team toward genuine CI?
A nightly build is not CI — it is a scheduled batch process. The key difference is feedback latency: a nightly build means a change committed at 9am gets no feedback until the following morning, by which time other developers have built on top of it. For an Revenue NZ project handling tax calculations, a defect that sits undetected for 18 hours can cascade into downstream data integrity issues. I would propose triggering the pipeline on every push, targeting sub-10-minute feedback, and enforcing branch protection so no one can merge a commit that fails the pipeline. The nightly report becomes the overnight extended test suite, not the primary quality gate.
Q8: What is the key difference between Continuous Integration and Continuous Delivery, and why do you need CI before CD can work?
CI ensures that every commit to main is automatically built and tested so the branch is always in a working state. Continuous Delivery extends that: every successful CI build is also automatically packaged and deployed to a staging environment, making the software releasable at any time. CD depends on CI because you cannot safely automate deployment of code that has not been continuously integrated and verified — deploying from a branch that has not been continuously tested just moves integration problems closer to production. CI is the prerequisite; CD is the extension.
Q9: A developer says "we can skip pre-commit hooks — the CI pipeline catches everything anyway, so running checks locally just slows down developers." What is wrong with this reasoning and how do you respond?
The reasoning treats CI as the only safety net, ignoring that CI feedback arrives minutes after a push — by which time the developer has moved on, a notification has interrupted them, and the context switch is expensive. Pre-commit hooks run in seconds, in the developer's IDE, before anything is pushed, so problems are caught at the cheapest possible moment. The cost of fixing a linting error before a push is near-zero; the cost of a broken build that blocks the team while the developer context-switches back is real. CI should be the second line of defence, not the first. Good CI hygiene is: local checks catch the fast stuff, the pipeline catches what local checks miss.
Why teams fail here
- Long-lived sprint branches — teams merge once per sprint at the end, using CI as a batch checker rather than a continuous integration gate, so the branch lives for two weeks and integration pain accumulates silently.
- Normalising red builds — once a team accepts that the pipeline is “usually orange”, the signal is gone; developers stop reading failure notifications and real regressions hide in the noise for days.
- Pipeline performance neglect — build times creep past 20–30 minutes as the test suite grows and nobody owns speed; developers respond by pushing less often, which is the opposite of the intended behaviour.
- Skipping CI with
--no-verifyor force-pushes under delivery pressure — this destroys the social contract of CI; once it is acceptable to bypass the pipeline “just this once”, it becomes the norm whenever a deadline approaches.
Key takeaway
Continuous Integration done well is not a tool you install — it is a team agreement that the main branch is always green, and that keeping it green is more important than any individual feature in progress.
How this has changed
The field moved. Here is how Continuous Integration evolved from its origins to current practice.
Grady Booch first uses the term "continuous integration." The concept is present but not yet a practice: integrate frequently rather than at the end of a long development phase.
Martin Fowler publishes the foundational CI essay. XP codifies CI: every developer integrates into the main branch at least daily. Automated build and test on integration is the key discipline.
CruiseControl and Hudson (later Jenkins) open-source CI servers make automated builds accessible to any team. The "broken build stops the line" culture begins.
GitHub Actions, Travis CI, and CircleCI make cloud-hosted CI available without infrastructure. CI becomes the default for open-source projects and spreads to enterprise development.
CI pipelines run within seconds for incremental builds using caching, parallel execution, and test impact analysis. AI-powered CI tools predict build failures, identify flaky tests, and suggest which tests to run based on changed code.