Insights and best practices for Test Management using QMetry tooling

Continuous Testing and Management in BFSI

The BFSI sector has a number of specific challenges when it comes to software development, testing, and test management. For example:

• Extensive digital transformations (from legacy systems to web and mobile applications)
• Real-time processes and required 24/7 availability of systems
• Integration and Interoperability: Apps must integrate seamlessly with several other applications
• Security of transactions
• Straight Through Processing (STP): transactions that can be processed automatically without human intervention
• Many projects and teams (and fragmentation of knowledge)
• Regulation and compliance: a great need for reporting and auditing options
• A balance between standardization of processes and tooling on the one hand and the freedom to empower teams to make their own decisions on the other.

These challenges have a major impact on the time-to-market of applications and the associated test processes. Testing must be executed faster, while the importance of quality control further increases.
In contrast to the "traditional" method (waterfall) of testing with strict phasing and many manual transfer moments, nowadays often other principles apply such as the modern testing principles and the concept of Continuous Testing (CT).

Studies from Forrester and DORA (DevOps Research and Assessment) show that CT is significant for the success of digital transformation proposals and initiatives. But what is CT all about? It can be narrowed down to the following mantra:  "testing early, testing often, testing comprehensively and automate."

This "Shift Left" approach often implies a different way of thinking and working including the use of tools around Continuous Integration / Continuous Delivery. In addition, CT also stands for a culture of close cooperation between teams and shared quality responsibility. This shift in the working method and mindset also increases the need for better communication, visibility, and collaboration around software quality. 

Based on my own experience and findings of a Dutch bank, I can share some recommendations and best practices around implementing continuous testing.

Best practices

In general, implementing a tool or a new way of working is a complex process and there is no silver bullet. However, there are some aspects to consider in terms of governance, continuity, and optimization.

Governance

Governance is about having complete ownership of the tools, with clear guidelines and (support) processes.

• Think big, act small: instead of implementing continuous testing across the board, start small and identify a single product or group which you can use to adopt the practice and one that has room for some trial and error. Once you have implemented it successfully, the example will serve you across the business.
• Build up the knowledge: having frequent feedback sessions to enable knowledge sharing allows you to build up an ecosystem of knowledge holders. Think about: walk-in sessions, demos, Train the Trainer meetings, etc.

Continuity

 The subject of Continuity is about having environments and applications that are stable, maintainable, and scalable.

• Don't over-automate: proper automation is a must-have, but don't over-automate. Know when to automate your tests and how to build your framework so that it quickly provides insight into the product's features. "Bad" automation suites tend to slow down teams.
• Infrastructure as code: businesses need to have the right infrastructure to provide environments quickly. In order to provide and manage IT infrastructure, tools like Ansible help to automate the infrastructure, making it easier to manage large volumes of infrastructure easily.

Optimization

Tools should follow a defined process. I feel that optimizing the tools to better fit your processes makes users much happier.

• Defined new Way of Working: it's very likely that users are used to a certain (old) way of working. To gently steer them in the new direction, use simple templates and guidelines for new processes and reconfigure tooling to the needs of the teams.
• Share the strategy: as information and knowledge can be scattered around the organization, it's important to share goals, roadmaps, and strategies in an easy-to-access and findable way (use Confluence for example)

Continuous Testing cannot live without clear visibility and transparency around quality. That's where QMetry comes in with a combination of tools, methodologies, frameworks and best practices that enable agile teams to build, manage and deploy high-quality software faster, with confidence.

How does QMetry fit in?

To start, QMetry offers two types of solutions: a Jira add-on (for Cloud and Server) and a stand-alone version (SaaS and on-premise). In this blog, I will only address the latter.

To summarize: QMetry Test Management is an enterprise-grade tool that is easy to use with a very intuitive interface:

• 20+ readymade integrations that are very easy to integrate with very little or no effort
• Importing/syncing user stories with Jira Integration
• DevOps tools such as Jenkins and Bamboo, Maven and Gradle
• Automation frameworks/tools such as Selenium based, QMetry Automation Studio, HP UFT, TestNG, JUnit, Specflow, Sauce Labs, Perfecto.
• 150+ Rest Open APIs
• Compliance ready with features like eSignature, Audit trails accessible through logs, general settings to restrict access and impose control, user-level permissions for accessing different modules and many more
• Leverage analytics with customizable dashboards, comprehensive reports, and advanced visual analytics
• Exploratory Testing – Save time by recorded test sessions with bug information and automated documentation

QMetry helps teams achieve Continuous Testing by integrating with the CI/CD pipeline along with capturing and monitoring the test executions.

Let's look at some feature examples with regard to some of the BFSI challenges:

Compliance with eSignature and Approval Workflows

QMetry’s eSignature helps to regulate the Approval Workflow of test cases and test executions to produce evidence that can be used for audit compliance. With the help of “Approval Workflow Report” teams can access summarized details of Test case approval and Test Execution approval.

Audit Logs and Version History

In an agile environment, Version history and Audit logs allow teams and management to view who has changed what and how many versions were created for a particular asset. With QMetry, you can monitor the evolution of test cases and requirements through version control as well as Audit Logs.

Integrations

The QMetry Digital Quality Platform offers seamless integration with different tools that form part of the CI/CD orchestration. Along with the built-in capability of integration with CI/CD and automation tools, QMetry Open API enhances the integration ecosystem via RESTful API.

Insightful Analytics

The success of Continuous Testing depends heavily on the efficiency of the feedback loops throughout the development and delivery cycle. QMetry Test Management provides extensive reports and analytics that are required for different members of the agile teams.

Test Automation / BDD

An important best practice in continuous testing is to have fully automated tests right from the design through production aiding the CI/CD environment. To achieve this, enterprise teams automate tests using Behavior-Driven Development (BDD). With QMetry, teams practicing BDD can author, store and sync feature files directly to/from Version Control Systems (VCS) like SVN, Git, and Bitbucket, using the built-in Gherkin editor in QMetry requirements. The two-way syncing between QMetry and version control systems keeps developers, testers and product owners up-to-date with the latest source code change commits with automated step definitions. Using QMetry automation agents, test executions can be triggered directly from QMetry. The Test Suite stores automation execution results ensuring complete feature traceability with requirements.

Feedback Loops

The efficiency of feedback loops throughout the development and delivery cycle is a big contributor to Continuous Testing success. QMetry Test Management provides extensive reports and analytics that are required by agile teams and their various members. Actionable insights in the form of dashboards enable management to have a bird’s eye view of the testing and product readiness. Some common report types in QMetry are:

• Requirement report showing the test coverage.
• Issue Report showing issue analysis and trends.
• Execution report showing execution per release cycle.
• Reusability Report is provided with the purpose of boosting the efficiency of the testing team with traceability, reusability, and visible linkages with other test assets.
• Traceability Report providing the trail of Requirements - Test Cases - Issues.
  

   

What are your challenges for Continuous Testing?

Every organization has its own challenges. We are happy to discuss how we can introduce and optimize Test Management within your organization. We have the relevant industry knowledge to guide you on this path, read more here about our QA Solution.

Get in touch!