Part 2 - SLA Goals using elapsed minutes instead of resolution times
Door Avisi / aug 2020 / 1 Min
Door Avisi / / 5 min
As described in my previous blog about Quality Assurance (QA), there are many challenges that need to be addressed when it comes to test management in an Agile world. As emphasized in The State of Testing Report 2019, one of those challenges is choosing the right tool(s) to use for test management.
It is easy to get overwhelmed by all the different vendors and the range of apps, especially with the variety of options available. This makes it hard to compare functionalities and unfortunately, there is no silver bullet for finding the best solution. It depends greatly on the nature of your testing activities and the key features needed, such as:
There are all kinds of possible variations, but regardless of what your business requires, a Test Management solution can help to increase either test efficiency, control over testing, or the quality of deliverables. This blog will compare the different approaches to finding a test management solution that fits your situation and needs.
Originally, JIRA Software was not designed to serve as a tool for test management. Though JIRA has several different issue types by default, there are no specific ones for testing. In addition, JIRA doesn't offer direct integration with automated testing frameworks, but you can customize JIRA in certain ways to use it for (manual) testing.
It's common in Agile projects to create a Scrum software development project and customize the configuration to the specific needs of the team. For example, a very basic configuration is to use the issue type 'Story', enter the acceptance criteria to which the user story should be validated and tested, and create subtasks for all work that has to be done within the sprint, including QA. It's good practice to create some new (sub-task) issue types in order to distinguish the specific type of QA work (for reporting and/or alternative workflows): 'Test Case' and 'Bug Task'. The regular Bug issue type is still being used, but only for bugs not related to a specific user story (e.g.: regression testing or non-functional testing). In those cases, bugs can be linked to the user story by using issue links.
![]() |
1. An example of customizing JIRA for testing |
It's also recommended to create a new screen for the Test Case issue type with some custom fields:
Depending on your team(s) specific way of working, you can choose to make a custom workflow for the Test Case issue type with different statuses (Passed / Failed).
It seems pretty easy to customize and set up JIRA for testing. While it could work in theory, it presents several challenges in reality:
It’s certainly possible to customize JIRA for Test Management. The downside is that it may require a lot of work. If you don’t have experience as a JIRA administrator, you will need to set aside several hours to configure JIRA to work for test case management.
So what are other options available if you want more control over your QA process? Integrations can help!
There are a lot of Test Management apps available via the Atlassian Marketplace. These apps can extend the functionality of JIRA and make it better equipped for test case management. But how to choose the right one for your business needs? It all depends on the context and the needs of your organization and team(s), like:
Based on these considerations (and possibly more), you can start the journey to finding the right solution for your needs. From the perspective of manual testing, many apps provide similar features for designing, planning and executing test cases. The biggest differences are in their user interface, the use of issue types, dashboard gadgets, built-in reports and support for automated tests.
As tool selection (and implementation) are important, we can help you with making the right choice. The following table lists a few popular apps that offer great functionality for basic test management (planning, scheduling, estimating, monitoring, reporting)) and more!
|
App
|
Our favorite features
|
---|---|---|
![]()
|
TM4J - Test Management for Jira |
|
![]()
|
QMetry Test Management for Jira |
|
![]()
|
Xray Test Management for Jira |
|
![]()
|
Zephyr for Jira - Test Management |
|
Using apps can help a lot with getting more control over your QA process. But you’ll still be a bit strapped in terms of what you can and can’t do as the app functionality is limited based on what the (JIRA) infrastructure allows it to do. When it comes to scaling, when teams grow from a large to an enterprise level (which demands customization, automation, and different testing types), you may be better off choosing a dedicated Test Management solution.
Next to (JIRA) apps, there are several other (dedicated) Test Management solutions available that provide more comprehensive functionality around advanced test case management. For example:
As there are many commercial and a few open-source tools available, we have narrowed down the list to three commercial tools that integrate with Jira and are available on both Cloud and On-premise:
App | Our favorite features | |
![]()
|
Zephyr Standalone |
|
|
QMetry Test Management |
|
|
qTest |
Choosing and implementing the right testing tools is important. The success of the implementation depends on both the selection of a tool to address the required improvement and the tool’s implementation process. Once a (thorough) selection and implementation process have been carried out, an adequate test tool (suite) will support the test process.
Approach
|
Pro's
|
Con's
|
---|---|---|
Customization in JIRA |
|
|
Integrating JIRA with Test Management apps |
|
|
Dedicated Test Management Solution |
|
|
Ultimately, which solution (JIRA only, JIRA + an app, or JIRA + an external integration) is right for your team depends a lot on the specific context and needs. We can guide you on this journey so that tooling and processes are complementary.
| Testing
Door Avisi / dec 2023
Dan denken we dat dit ook wat voor jou is.