AI and testing are a match made in heaven
September 20, 2017
As the scope of testing is increasing and the time available to test is shrinking, it?s clear that teams need a new, modern approach that enables true, end-to-end test automation.
You can’t escape the hype around artificial intelligence (AI) and how it’s going to change the way we work and live. Nothing appears to be safe from its clutches, but despite some doom-and-gloom scenarios, software testing is one area that’s already seeing positive benefits from AI’s transformative powers.
DevOps, continuous delivery, and agile are reducing cycles from years or months to weeks or days, giving teams less time to test. It’s extremely difficult for test teams to keep up with that pace using testing approaches from over 15 years ago. Mobile, web, Internet of Things (IoT), and technology fragmentation in general mean that, while everyone used to test on just one platform, they’re now testing 40 or more. To get greater coverage of user journeys through their app, teams are finding that hundreds of tests aren’t adequate—they need thousands of tests.
So, as the scope of testing is increasing and the time available to test is shrinking, it’s clear that teams need a new, modern approach that enables true, end-to-end test automation. AI, machine learning, and analytics will be the catalysts to deliver true test automation — recommending the tests to carry out, learning continuously, intelligently predicting business impacts, and enabling development teams to fix issues before they occur.
Three ways AI is enabling predictive testing
1. Intelligent automation. The only way to truly test a digital app from the user perspective is through an intelligent automation engine that accesses the application as a user would—taking control of a machine, actually using the app to exercise workflows, and collecting intelligent analytics along the way. This requires technology that understands on-screen images and text, such as smart image search and dynamic neural networks (so-called deep learning).
2. Intelligent test coverage generation and bug-hunting. There is a potentially infinite number of paths through a complex app, so which ones should we follow in our automation? We can use AI classification algorithms, such as Bayesian networks, to select paths and bug hunt. As these paths are explored, the bug-hunting AI algorithm continues to learn from correlations in data to refine the coverage and help developers identify root causes and fix defects.
Through a combination of bug hunting and coverage algorithms, AI and analytics will exponentially increase coverage and productivity. AI algorithms will hunt for defects in applications based on user journeys that are automatically generated from this bug-hunting model, while coverage algorithms will select the user journey that is the furthest away from others that have already been executed.
Non-instance-based learning algorithms also reduce the amount of learning required, giving quick results, which are essential in agile and DevOps environments. Ensuring that the algorithms are delivering defects and coverage is a balancing act, and where AI will need input from a smart tester who knows the system well and can dynamically adjust the trade-off between coverage and bug hunting.
3. Continuous testing, continuous learning, and predictive trends. Testing digital apps is not just a one-and-done exercise. It should be a continuous process, so that we’re essentially monitoring the digital experience over time. An AI algorithm should be watching the test results over time, learning and looking for trends. These learning algorithms can then build decision trees that enable predictive analytics — and can identify, for example, if, based on experience, the increasing delay on a particular workflow will likely result in a system outage. Armed with this information, we could proactively address it before it becomes critical and causes customer outrage.
The transformative impact of AI will fundamentally change what testing is all about. Testing will move closer to the user. Through predictive analytics, testing will move closer to product design. And by learning the relationship between technical behaviors and user satisfaction, conversion, and retention, testing will move closer to revenue. AI will enable testing to quickly be perceived as a revenue-generating profit center rather than a opaque overhead.
