Remember the first blog topic about ‘The Essence of Software Testing’? After nearly a year, I have a preliminary answer: “Avoiding Risk”. The fundamental reason for not testing is choosing to ‘accept the risk’.
Can’t test everything? Release as is due to fixed project timelines. 500 regression tests? Just test as much as possible. Need more testers? Let’s hold off and burden the developers.
These examples illustrate forced risk acceptance scenarios.
Under the principle of ‘avoiding risk’, how do we ensure our product is bug-free? There are two starting points:
- Reducing bug generation.
- Discovering bugs.
But before tackling these, we must define what a bug is—a philosophical question harder than my postgraduate studies in Epistemology.
For instance, ChatGPT was reported to have a bug because it couldn’t limit responses to ten words. But is the bug from ChatGPT or user expectations? If a product fails to meet user expectations, it’s often labeled a bug, ignoring that users vary greatly.
Why are users moving to competitors? Is it due to bugs affecting our speed and stability? Why do many users find our product slow? Is it their internet or something else? If we can’t break through the Great Firewall, we can’t reach Mainland China users. Is that a bug?
Since defining a bug is elusive, let’s return to the two starting points:
Reducing Bug Generation: From a developer’s perspective, how to avoid bugs while coding. Famous methods include unit testing, pair programming, code reviews, and static code analysis.
Discovering Bugs: From a tester’s perspective, methods like test left-shifting, automated regression testing, and manual exhaustive testing of daily builds are prominent.
Investment in Testing: Assess your company’s investment in testing from manpower, time, and financial perspectives. This will lead to a realization I’ll discuss next: “Investment rarely prioritizes testing.” Despite product issues, companies focus on product development over perfecting existing ones.
Recursive Nature of Testing: Even if AI is invested in testing, the issue of testing’s recursiveness arises, especially in automated testing. Who tests the testers’ tools? This recursion leads back to investment issues.
Testability: Accepting the issue of recursiveness, testability remains a challenge. Different technologies behind similar user experiences mean different testing approaches. Testers need to understand the product better than PMs and RDs. In scenarios where testability is unachievable, human intervention remains crucial.
These three issues form a cycle. Automation or AI reduces manpower to an extent, but increased complexity and costs arise with each reduction. The diminishing marginal benefits eventually halt investment, leaving things in limbo.
Thus, I promote semi-automation with human intervention over full automation. When questioned about human error, remember: users are humans. Even if robots replace many tasks, we can still become engineers who repair them. Remember, “Every bug creates job opportunities.”
Instead of worrying about AI replacing software testers in ten years, consider how long it’ll take for automation engineers to fully automate their work.
