Xray
The most powerful test management app for Jira. BDD native, automation-ready, and trusted by thousands of agile teams worldwide.
Overview
Xray is a test management app for Jira that provides comprehensive test case management, test execution, and reporting within the Jira interface. Unlike Zephyr Scale which focuses on simplicity, Xray is designed for teams that need advanced test management capabilities — BDD support, automation integration, requirements traceability, and detailed analytics.
Xray is the most popular Jira test management app, with over 5,000 customers worldwide. It supports manual testing, automated testing (via CI integration), and BDD workflows with Cucumber and Behave.
What it's used for
Xray is ideal when:
- Advanced Jira test management needed: More powerful than Zephyr Scale with better reporting and traceability.
- BDD workflows: Native Cucumber support with automatic step definition linking.
- Automation integration: Import results from Playwright, Selenium, JUnit, TestNG, and more.
- Requirements traceability: Full coverage matrix from requirements to test cases to defects.
Pros & Cons
Pros
- Most powerful Jira test management app
- Excellent BDD support with Cucumber/Behave
- Strong automation integration with CI tools
- Detailed traceability and coverage reporting
- Active community and regular updates
Cons
- More expensive than Zephyr Scale
- Can be complex to configure for simple use cases
- Jira-only — no standalone option
- Can impact Jira performance with large test repositories
- Steeper learning curve than Zephyr Scale
Platforms & Integrations
Xray is a Jira app available for Jira Cloud and Jira Data Centre. It requires an active Jira subscription.
Pricing
| Tier | Cost | Includes |
|---|---|---|
| Jira Cloud | From $5/user/mo | Full features, cloud hosting, Xray support |
| Jira Data Centre | Custom | Self-hosted, perpetual licence, annual maintenance |
NZ Context
Xray is the most popular Jira test management app among NZ agile teams with mature QA processes. It is frequently mentioned in NZ testing meetups and training courses. For NZ teams using BDD (Cucumber/Behave), Xray is the natural choice. Job postings for senior QA roles in NZ increasingly list Xray as a preferred skill.
Alternatives
- Zephyr Scale — Simpler and cheaper Jira test management.
- TestRail — Standalone tool with richer reporting.
- qTest — Enterprise alternative with Tricentis integration.
When to choose Xray
A quick decision guide for NZ teams evaluating test management for jira options.
| Choose Xray when… | Choose something else when… | Combine with… |
|---|---|---|
| Your team already runs Jira and writes Cucumber feature files checked into Git | Your Jira instance is shared across hundreds of users and performance is already a concern — Xray adds index overhead; consider TestRail standalone instead | Playwright or Cypress for automated execution — pipe JUnit XML results into Xray via the REST API to close the loop on coverage |
| You need a full requirements-to-defect traceability matrix for audits or regulated delivery (financial services, health) | Your team is small (fewer than 5 testers) and runs mostly exploratory or ad-hoc testing — Zephyr Scale is simpler, cheaper, and faster to set up | Confluence for test plans and strategy documents — Xray links Jira tickets but Confluence carries the narrative context that stakeholders actually read |
| Senior QA hires are expected to bring BDD experience and you want the tooling to match what they already know | You are not on Jira and have no plans to migrate — Xray has no standalone offering; qTest or PractiTest cover the same traceability without the Jira dependency | GitHub Actions or Jenkins — trigger test runs on PR merge, publish results back to Xray automatically so coverage data stays current without manual uploads |
| CI pipeline already produces JUnit, TestNG, or Robot Framework XML and you want results visible inside Jira without a separate dashboard | Budget is the primary constraint — Xray adds $5+/user/month on top of Jira costs; Zephyr Scale is marginally cheaper and adequate for structured manual test cycles | Allure Report for a richer visual report for developers — use Xray for the Jira coverage view and Allure for the test-run HTML that developers prefer to open |
What I would do
Practitioner judgment on tool adoption, team onboarding, and when to swap.
If…
I were the QA lead at CloudBooks, where squads run Playwright suites on every PR and BDD acceptance criteria are authored alongside stories in Jira
I would…
Adopt Xray and spend the first two weeks on one thing only: the CI upload pipeline. Get JUnit XML flowing into Xray automatically so coverage percentages update on every merge. Once that's green, add the requirements-to-test link so the coverage matrix writes itself. Do not ask testers to manually update Xray execution results — that behaviour kills adoption inside a month.
If…
I were joining the QA chapter at Harbour Bank, where the platform team has flagged Jira performance degradation and there's an active conversation about moving some tooling off Atlassian
I would…
Not push for Xray. The index overhead is real when you're running tens of thousands of test cases across multiple programmes. I'd evaluate TestRail instead — it sits outside Jira, keeps test management performance independent, and integrates back into Jira via webhooks for defect linking. If the organisation later consolidates onto Atlassian for the long haul, revisiting Xray then makes sense.
If…
I were setting up a QA practice from scratch at TechServNZ, onboarding a mixed team of manual testers moving into automation and fresh graduates who have never used a formal test management tool
I would…
Start with Zephyr Scale for the first six months. Get the team comfortable writing structured test cases and executing against them before introducing BDD or automation. Once they understand what a test cycle is and why coverage matters, the jump to Xray is logical rather than overwhelming. Switching costs are low — Xray can import Zephyr test cases — so you lose nothing by starting simpler.
The bottom line: Xray earns its cost only when the CI pipeline writes results back automatically — if your team is updating execution status by hand, you have bought an expensive spreadsheet inside Jira.
Interview questions
Questions you are likely to get if you list Xray on your CV — with what interviewers are really testing for.
What is the difference between a Test in Xray and a Test Execution? When would you create each?
What they’re really testing: Whether you understand Xray’s core data model or have only clicked around the UI without grasping the separation between test definition and test run.
Strong answer covers: A Test is a reusable artefact (steps, preconditions, expected results) that lives permanently in Jira; a Test Execution is a container that links one or more Tests to a specific build or sprint and records actual pass/fail results for that run — analogous to a test plan execution in other tools. Mention that one Test can appear in many Executions across releases, and that this separation is what enables Xray’s coverage matrix to track which requirement was last verified against which build.
When would you choose Xray over Zephyr Scale for a Jira-based team?
What they’re really testing: Whether you can make a reasoned tool selection rather than defaulting to whatever you last used.
Strong answer covers: Choose Xray when the team writes Cucumber or Behave feature files (Xray has native BDD support that Zephyr Scale lacks), when CI pipeline results need to flow back into Jira automatically, or when a full requirements-to-defect traceability matrix is required for audit or regulated delivery. Choose Zephyr Scale when the team is smaller, mostly manual, or needs a faster setup with a gentler learning curve. Note the cost difference — Xray is more expensive — and that in the NZ market you often encounter Xray on mature agile teams in financial services or telco where traceability is non-negotiable.
You’re joining the QA chapter at TeleNZ. The team runs Playwright suites in GitHub Actions and wants test results visible inside Jira without manual uploads. How would you wire that up with Xray?
What they’re really testing: Whether you can describe a real CI integration end-to-end, not just say “Xray supports automation”.
Strong answer covers: Configure Playwright to output JUnit XML (via the junit reporter); add a GitHub Actions step after the test run that calls the Xray REST API endpoint (POST /rest/raven/2.0/import/execution/junit) with the XML file and a target Test Plan key; Xray creates a Test Execution automatically, links results to existing Test issues by name match, and updates the coverage dashboard. Mention that you’d store the Xray client ID and secret as GitHub Actions secrets, and that on Xray Cloud you authenticate via the /authenticate endpoint to get a short-lived Bearer token before each upload.
Your Playwright tests pass locally but fail in CI, and the Xray Test Execution shows them all as “FAIL” even though the HTML report looks green. How do you investigate?
What they’re really testing: Whether you understand that Xray only reflects what the uploaded results file says, and can reason about where the disconnect is.
Strong answer covers: First confirm the JUnit XML was actually generated by Playwright for that CI run (not a cached file from a previous run); open the XML and check that test names match the Xray Test issue summaries exactly — a mismatch causes Xray to create new unlinked Tests rather than update existing ones. Check the Xray import response body in the CI logs for warnings. Also verify the Test Execution was created against the correct Test Plan and project key. A common NZ-context gotcha: Jira Cloud rate-limits the Xray REST API, so parallel CI pipelines can hit 429 errors mid-upload and produce a partial or corrupt Execution.
How would you structure Xray to support a large programme at a NZ organisation like HealthNZ, where multiple squads own different service domains and share a single Jira project?
What they’re really testing: Whether you can think about Xray governance and scale, not just individual test case creation.
Strong answer covers: Use Test Plans to scope coverage by squad or domain (e.g. Referrals, Scheduling, Pharmacy); use Jira labels or components to tag Tests by domain so the coverage matrix can be filtered per squad without needing separate projects. Use Test Sets to group reusable regression Tests independently of any single Execution, so squads can clone a regression set for each sprint without duplicating Test definitions. Establish a naming convention for Test Execution summaries (e.g. [SQUAD]-[SPRINT]-[ENV]) so the programme manager can scan execution history at a glance. Mention that on Jira Cloud a single Xray instance spans the whole org, so governance around who can create or archive Tests matters more as the repository grows.