This is a rather special time in the evolution of software testing. Teams worldwide are facing new challenges associated with working from home. Digital transformation initiatives are placing unprecedented pressure on innovation. Speed is the new currency for software development and testing. The penalty for software failure is high as news of outages and end-user frustration goes viral on social media. Open-source point tools are good at steering interfaces but are not a complete solution for test automation. Meanwhile, testers are being asked to do more while reducing costs.
Now is the time to re-think the software testing life cycle with an eye towards more comprehensive automation. Testing organizations need a platform that enables incremental process improvement, and data curated to optimize software testing must be at the center of this solution. Organizations that leverage multiple open-source or proprietary testing tools must consider an Open Testing Platform to keep pace with Agile and enterprise DevOps initiatives.
What is an Open Testing Platform?
An Open Testing Platform (OTP) is a collaboration hub that assists testers in keeping pace with change. It transforms observations into action – enabling organizations to inform testers about the critical environment and system changes, act upon comments to zero in on ‘what’ precisely needs to be tested, and automate the acquisition of test data required for adequate test coverage.
RELATED CONTENT:
Testing tools deliver quality – NOT!
The de-evolution of software testing
The most important feature of an Open Testing Platform is that it taps essential information across the application development and delivery ecosystem to test software effectively. An OTP allows any tester (technical or non-technical) to access data, correlate observations, and automate action. Beyond accessing an API, an OTP leverages an organization’s existing infrastructure tools without causing disruption—unlocking valuable data across the infrastructure.
Model in the middle
At the core of an Open Testing, Platform is a model. The model is an abstract representation of the business’s strategic transactions. The model can represent new user stories in-flight, system transactions critical for business continuity, and pivotal flows for the end-user experience.
In an OTP, the model is also the centerpiece for collaboration. All tasks and data observations either optimize the model’s value or ensure that the tests generated from the model can execute without interruption. Since an OTP is focused on the software testing life cycle, we can use known usage patterns and create workflows to accelerate testing. For example, with a stable model at the core of the testing activity:
- The impact of change is visualized and shared across teams
- The demand for test data is established by the model and reused for team members
- The validation data sets fit the logic identified by the model
- The prioritization of test runs can dynamically include each team’s process stage, optimizing for vectors such as speed, change, business risk, maintenance, etc.
Models allow teams to identify critical change impacts quickly and visually. And since models express test logic abstracted from independent applications or services, they also provide context to help testers collaborate across team boundaries.
Data curated for testing software.
Data must drive automation. An infrastructure that can access real-time observations and reference a historical baseline is required to understand the impact of change. Accessing data within the software testing life cycle is not intrusive or depends on a complex array of proprietary agents deployed across an environment. In most cases, accessing data via an API provides enough depth and detail to achieve significant productivity gains. Furthermore, accessing data via an API from the current monitoring or management infrastructure systems eliminates the need for additional scripts or code that require maintenance and interfere with overall system performance.
Many of the data points required to optimize the process of testing exist. Still, they are scattered across various monitoring and infrastructure management tools such as Application Performance Monitoring (APM), Version Control, Agile Requirements Management, Test Management, Web Analytics, Defect Management, API Management, etc.
An Open Testing Platform curates data for software testing by applying known patterns and machine learning to expose change. This new learning system turns observations into action to improve testing effectiveness and accelerate release cycles.
Why is an Open Testing Platform required today?
Despite industry leaders trying to posture software testing as value-added, the fact is that an overwhelming majority of organizations identify testing as a cost center. The software testing life cycle is a rich target for automation since any costs eliminated from testing can be leveraged for more innovative initiatives.
If you look at industry trends in automation for software testing, automating test case development hovers around 30%. If you assess the level of automation across all facets of the software testing life cycle, then automation averages about 20%. This low average automation rate highlights that testing still requires a high degree of manual intervention, slows the software testing process, and delays software release cycles.
But why have automation rates remained so low for software testing when initiatives like DevOps have focused on accelerating the release cycle? Four core issues have impacted automation rates:
- Years of outsourcing depleted internal testing skills
- Testers had limited access to critical information
- Test tools created siloes
- The environmental changes hampered the automation
Outsourcing depleted internal testing skills
The general concept is that senior managers traded domestic, internal expertise in business and testing processes for offshore labor, reducing Opex. With this practice, known as labor arbitrage, an organization could reduce headcount and shift the responsibility for software testing to an army of outsourced resources trained on software testing. This shift to outsourcing had three significant detrimental impacts on software testing: the model promoted manual task execution, the adoption of automation was sidelined, and there was a business process “brain-drain” or knowledge drain.
With the expansion of Agile and adopting enterprise DevOps, organizations must execute the software testing life cycle rapidly and effectively. Organizations will need to consider tightly integrating the software testing life cycle within the development cycle, which will challenge organizations using an offshore model for testing. The team must also think beyond the simple bottom-up approach to testing and re-invent the software testing life cycle to meet the increasing demands of the business.
Testers had limited access to critical information
Perhaps the greatest challenge facing individuals responsible for software testing is staying informed about the change. This can be requirements-driven changes of dependent applications or services, changes in usage patterns, or late changes in the release plan, impacting the testers’ ability to react within the required timelines.
Interestingly, most data required for testers to do their job is available in the monitoring and infrastructure management tools across production and pre-production. Access to APIs and advancements in managing and analyzing significant data changes this dynamic in favor of testers. However, this information just isn’t aggregated and optimized for software testing.
Although each organization is structurally and culturally unique, the one commonality among Agile teams is that testing software has become siloed. The silo is usually constrained to the group or a single application built by multiple teams. These constraints create barriers since tests must execute across componentized and distributed system architectures.
Ubiquitous access to best-of-breed open-source and proprietary tools also contributed to these silos. Point tools became very good at driving automated tests. However, test logic became trapped as scripts across an array of devices. Giving self-governing teams the freedom to adopt a wide variety of tools comes at a cost: a significant degree of redundancy, limited understanding of coverage across silos, and a high amount of test maintenance.
The good news is that point tools (open-source and proprietary) have become reliable in driving automation. However, what’s missing today is an Open Testing Platform that assists in driving productivity across teams and their independent testing tools.
The environmental changes hampered automation.
Remarkably, the automated development of tests hovers at about 30%, but the automated execution of tests is half the rate at 15%. This means that tests that are built to be automated are not likely to be executed automatically – manual intervention is still required. Why? It takes more than just automation to steer a test for automation to yield results. For an automated test to run automatically, you need the following:
- Access to a test environment
- A clean environment configured specifically for the scope of tests to be executed
- Access to compliant test data
- Validation assertions synchronized for the test data and logic
As a result, individuals responsible for testing need awareness of broader environment data points located throughout the pre-production environment. Test automation will continue to have anemic results without automating the sub-tasks across the software testing life cycle.
An Open Testing Platform levels the playing field.
As systems become more distributed and complex, the challenges associated with testing compounds. Despite the hampered evolution of test automation, testers, and software development engineers in test (SDETs) are being asked to do more than ever before. Yet, the same individuals are under pressure to support new applications and technologies while facing a distinct increase in the frequency of application changes and releases. Something has got to change.
An Open Testing Platform gives software testers the information and workflow automation tools to make open-source and proprietary testing point tools more productive in light of constant change. An OTP provides a layer of abstraction on top of the teams’ point testing tools, optimizing the sub-tasks required to generate suitable test scripts or no-code tests. This approach gives organizations great flexibility while significantly lowering the cost to construct and maintain tests.
An Open Testing Platform is a critical enabler to both the speed and effectiveness of testing. The OTP follows a prescriptive pattern to assist an organization in continuously improving the software testing life cycle. This pattern is ‘inform, act and automate.’ An OTP offers immediate value to an organization by giving teams the missing infrastructure to manage change effectively.
