The end of QA as we know it

QA is the quality compass of the company, keeping the big bugs away from the customers as well as pushing the developers to create better products.

However, in the past several years more and more products are shifted towards the cloud. New functionality is delivered to production each week, day or even commit. This state leaves no room for the old fashion QA, forcing a major change in the approach for handling quality.

The third revolution of QA

The birth of QA

At the dawn of software development, there was no real quality control. The developers wrote code and ran it, there was no dedicated person to verify that the code they wrote actually did what it's supposed to do.

Eventually, we came to an understanding that the product should be tested before it's shipped to the customer, preferably by someone else other than the developer who built it.

This was the first revolution, the one that gave birth to QA!

At those days, it was a necessity. Most developers don't like to test their code. They prefer to write features rather than write unit tests or run regression, so there was a need for someone to take responsibility over the quality of the developed product.

In addition, this came in a perfect harmony with the Waterfall development methodology, setting QA as the gatekeeper for the release of the product.

The agile QA

The second revolution happened when the Agile methodologies started to dominate software development.

One of my former managers lent me a book called Agile Testing: A Practical Guide for Testers and Agile Teams by Lisa Crispin.

Although it wasn't a light reading, it bared a fascinating new approach for the core definition of QA. Basically, it redefined the role of a tester from being someone that starts his work only after the development is done, into an integral part of the development process. The tester was encouraged to be more involved in the development stages, asking questions about both the design and the code from a quality perspective.

That was a very painful revolution, especially for the developers. The fact that testers, which were considered much lesser compared to developers (a crucial mistake that hunts us until today), were now sitting as equals in the design and spec sessions, was truly an abomination.

The Cloud

The third revolution started when software products shifted towards the cloud. Building cloud services changed the way we architect and develop software. Instead of major releases we are constantly delivering features and bug fixes to production. Instead of a single product shipped on a disk or as an installer, there are swarms of micro-services running on VMs. This shattered yet again the core foundation of QA, and this is what this article is all about.

---

The end of QA as we know it

No room for QA

The life cycle of a cloud service can be illustrated by a simple diagram:

The main goal of any successful SaaS operation is to minimize the time that this cycle takes.

Just think about it, if you are able to detect an issue, develop a solution and deploy it before the customer notices the problem then your process will be able to support any agility it requires.

But unfortunately, this ability leaves no time for any test reviews nor manual regression cycles.

Major issues are found only in production

Most of the major issues that happen in SaaS services are found only in production.

The main reason for that is a simple fact that the complexity of today's services is so high, that it's almost impossible to run all possible scenarios in a synthesized test environment.

Moreover, the amount of data, the load and the variety of user scenarios, sometimes caused by unintended usage, makes it utterly difficult to test every possible outcome.

Thus, the amount of critical bugs that are found during the QA phase is too insignificant compared to the amount found by analyzing telemetry or by customer complaints. Combined with the high price in time and resources that you pay in order to carry out those tests, makes you wonder whether or not this investment is really worth it.

But, who will stand guard?

The main reason why any management will afraid to invest in a fundamental change of QA is a plain fear of loosing a single quality gatekeeper. I agree that it's a bit frightening not to have a single guardian that you can blame in case the quality of the shipped product is bad.

But is it really the case?

Can QA be transformed into something better, like a butterfly hatching from it's cocoon?

I believe it can!

Most QA organizations are responsible for two domains: manual testing, and automation.

QA as Dev

When developing automation, there is no real difference between writing complex acceptance tests and writing a product. Building complex automation upon all it's aspects can be a product of it's own, that means it should be built by developers. The fact that there is a separation between a product code and automation test code is wrong in the first place. This state only makes the developers less accountable for the quality. It leaves room for shoving the responsibility for any bug towards an automation issue just because it provides an opportunity to blame someone else, and refrain from doing the work they don't like.

Moreover, because automation isn't part of the development organization it's much more likely to have looser coding and design standards.

Not once I've witnessed the dismissal of building a quality automation in order to save time.

Sounds a bit ironic isn't it?

QA as Product Owner

"Not all can be tested with automation! You need to get your hands dirty and run manual checks!"

This might be the biggest argument of all for still having manual QA.

But, is it really true?

Suppose that you have the most thorough automation, that provides regression coverage to almost any functional scenario. Also, it's backed by a comprehensive telemetry that can spot almost any issue in production.

Then the only thing left for you to check is that the developed content is in fact what was intended by the product owner (being a customer or product manager).

Actually, there is a famous comic sketch about it:

Ask yourself, how many times have you found yourself in a situation, when after numerous test cycles and countless spec meetings, the product that was developed and tested, was not the one intended by the product owner?

Only after it was deployed to production or presented to the customer, you discovered that it wasn't doing what it supposed to do.

Now imagine, that before delivering the customer verifies the product. These aren't extensive regression tests but a small verification that must be done manually in order to make sure that the delivered content is working correctly. In fact, that is the only thing you should verify manually, anything else will just add useless overhead and waste resources.

Post-Mortem

This article might seem a bit controversial, but it's only a mindset change.

Applying a fundamental change to QA is not a thing to take lightly.

It's a change of shifting the accountability over the quality from QA towards developers, and it requires a change in the way you perceive development.

In other words, in order to make this change possible you must provide more time for developers to invest in quality and be fully accountable for it, allowing them to truly own the the product they develop (read more: The notion of False Time Estimation Syndrome) .

Although it seems very expensive, in the long run you won't only save money but you'll also create a lean organization that is able to deliver quality product with minimal overhead.