It’s normal when the QA teams decide to shift to the test management tools that are in line with the requirements of JIRA when the development teams move towards JIRA from legacy ALM. Although this can prove to be a beneficial move for an organization running a small number of test cases, it can cause some significant performance issues for larger organizations that plan to increase levels of test automation – specifically with JIRA Server and JIRA Data Center.
These performance issues are mainly driven by the accumulation of data and space consumption of the app since JIRA apps lack built-in storage and all the data is stored on the JIRA database. This can make the entire JIRA instance vulnerable to performance degradation, crashes, and stalls, impacting every JIRA user throughout the organization in as little as 14 weeks, according to research.
When JIRA test management tools cause JIRA to crash, they restraint testers from reusing, accessing and storing test cases. As a result, all traceability and visibility are lost. And most of the time, these two are the main reasons why organizations implement test management in the first place.
To solve this scalability issue, you can pay for additional JIRA data center machines to continue storing and accessing a growing set of test management data. This can be a reasonable solution for manual shops, but it’s a different case for companies that are moving towards higher percentages of automated testing because the amount of data they will need to store will grow exponentially, requiring them to continue adding additional machines indefinitely.
To better understand where exactly the performance issues lie, we ran an experiment with a JIRA server and a common test management app. We found out that:
- Jira test management tools can be a viable solution for companies running manual test cases on a small scale since loading a few thousand test cases didn’t significantly affect the use of JIRA.
- The problem starts when the number of test executions increases to, for example, 100,000 where performance was significantly impacted. In fact, Jira crashed multiple times when loading execution results from a standard to a large-sized test suite.
- The JIRA API was significantly impacted overall and there was a drastic increase in the latency on the entire application affecting business analysts, developers, non-testers, and other users.
While we’ve taken our time to inform you about the issues, it certainly does not mean that acquiring good test management tools is not a good idea. In fact, there are certain qualities of these tools that can help you improve your testing processes and outcomes. Modern test management tools provide:
- A single view of all testing projects: See all your testing activities in one place.
- Real-time actionable insights: Comprehensive reports and metrics can provide unprecedented insights.
- Support for multi-speed IT: Adaptability allows it to go with the development methodology of your choice.
- Instant collaboration: Allows in-tool commenting and communication between stakeholders.
- Easy migration: Import all test cases from any legacy application to a modern test tool in just a few clicks.
