Should You Talk to Clients About Testing?

That is the dilemma that troubles developers and salespeople alike when they start a new project and define budgets, explaining to clients what this choice entails.

With this post I want to share Cantiere Creativo’s experience managing the web projects we’ve developed in recent years where we applied testing practices. I don’t have enlightening answers, just open reflections and a few tips born from mistakes we’ve made.

Whatever the size of the project, our goal is always “quality.” It’s a broad concept that cuts across many areas: quality in how we manage work, in office life, in aesthetics and, for development, in the code we write. To write quality code, we couldn’t do without tests, which for us means unit tests, integration tests, TDD and BDD. We explain these techniques in detail on a dedicated page.

This is the recurring question in our sales meetings. Over time we’ve tried different approaches and come to this view:

NO: If we don’t talk to the client about tests, in practice, the problem never comes up. Nor will the time and budget needed to write them be accounted for, because the client won’t fully understand the cost and time difference compared with competitors who don’t adopt this practice. Even if a developer started writing tests on their own initiative and out of necessity, they would soon have to give up (with sadness and frustration) for lack of time and resources.

So, simply put: if you don’t talk about them, tests don’t get written. Yo!

YES: Explaining what tests are is really hard, and in 90% of cases the client has no idea what we’re talking about. A few eager ones, fascinated by technology, may imagine extremely complex scenarios, fascinating but oversized compared to what they think they need.

Either way, during a detailed explanation, our listeners’ faces will most likely be full of skepticism, boredom and fear, and in some cases you might even notice a hint of disgust!

Editor’s note: I’m jokingly generalizing; in some cases I’ve seen clients — usually those with technical skills — happy to hear about tests!

As developers, we’re well aware that tested code benefits everyone: the client gets what they want, namely a working, verifiable product; the developer sleeps soundly after a deploy, and is less afraid of touching risky areas; in fact, the whole project benefits, because it can enjoy a long life on a solid foundation.

That’s why we decided to talk about tests with our clients.

We had to ask ourselves how to sell tests as an integral part of project development. After a few attempts, we found our formula.

TRAINING: client meetings became deep-dive sessions where we explain the agile approach and Scrum, going into the details of the method we follow. Tests are part of this necessary training.

QUALITY: we make it clear that internal quality is non-negotiable for us. It concerns the infrastructure and the code we build.

VIDEO: using videos that show a complete test run of an app helps us show visually what we’re talking about. Where a human test takes an hour or two with a high margin of error, the test suite does it in just over a minute, guaranteeing that every point, button and form is tested for correct behavior.

FULLY TESTED CODE: having 95% coverage means you can add new features without fearing regressions, that is, breaking features written earlier; it means having code that stays maintainable over time.

To make these benefits easier to grasp, we find it useful to show a chart comparing an approach with and without tests.

In the no-test option, initial development is certainly faster and cheaper, but every change will eventually bring us to a point of no return where any modification becomes tremendously expensive, if not impossible.

A tested application will have a slower, and therefore more expensive, startup phase, but the initial costs will be recouped over time because additional features won’t break what already exists.

Code documentation is very important; we also know that all too often it’s the first activity to be dropped as development progresses and budgets shrink. Tests fulfill this role perfectly, because they inherently describe what functions do.

Having a test suite helps when the development team grows, so a new dev won’t be afraid to “get their hands dirty” and play with the existing code; it also lets the client switch suppliers, which can happen for a thousand reasons. It may also happen that the project goes well and the client decides to build an in-house team to maintain it or develop other parts of the app. That day, you’ll look really good! ;)

So why does the dilemma exist? It all seems logical and wonderful... Where exactly is the problem?

To write tests, you need a clear idea of the expected behaviors. We know that clients, especially early in a project, never really know what they want and will change their minds a thousand times! But be careful: the problem isn’t the tests; on the contrary, tests bring the problem to the surface, make it visible, and can’t be written until it’s addressed and solved.

Every developer needs practice to get comfortable with the mindset where code starts from the constraint of tests and evolves from there. This certainly depends on the developer’s skills and experience, but even a very good developer who isn’t particularly familiar with TDD will find it extremely uncomfortable to get used to this way of reasoning.

Writing good tests is far from trivial. You have to test something that doesn’t exist yet without falling into the trap of pre-engineering the application. Over-engineering is also always lurking: by going into excessive detail about how something works, you risk generalizing too much, implementing behaviors and tests that aren’t actually required or needed.

Then you enter the magical world of doubling! Besides the application “you can see,” the test suite must be maintained too, and it’s made of code as well! If tests aren’t treated with the same care (or perhaps more), we risk placing our trust in a tool that is actually fragile.

Double the code. The test ratio is generally 2:1. If a test suite ends up with twice as much code as the app, and since tests are code too, where is the greatest risk of chaos breaking out? Care, care, care!

Double the budget. We have to write 200% more code, especially in the initial startup phase. That takes time, and therefore money!

We learn from our mistakes, and we never stop. As I said, I don’t have great answers to the dilemma, but from a management perspective we’ve formed some ideas. We’ve found there are two fundamental questions clients need to answer with complete transparency.

If the app doesn’t need future development, it’s born legacy, so we can say with some confidence that it won’t need tests. If it will change “a bit,” we need to understand exactly where and how much, so we can identify the most sensitive areas of the project.

This question seems even more obvious! Everyone answers “extremely,” nobody wants bugs anywhere. But bugs exist! Dig just a little and you’ll find areas of the application more sensitive than others, where a bug is unacceptable. Imagine a scenario involving payments or the analysis of important health-monitoring data. A bug there will weigh far more than one on the “About us” page. Prioritizing bugs, once again, helps us understand which areas of the project are most sensitive and must definitely be tested.

If a project needs tests, you need developers experienced with these practices. First attempts at writing tests are very likely to be unstable, and having a shaky suite in production is not desirable.

Clients perceive the value of tests from the fact that their app doesn’t break and stays solid over the months. Even if everything always goes well, over time that perception fades and clients tend to forget it. It’s worth periodically delivering reports generated by any tool built for the job, documenting the health of the code being created; clients won’t understand a thing, but they’ll still be happy to know their app is in good shape!

Be resolute! Clients will always want to save money, and sometimes we’re tempted to go along with the apparent savings. We must always remember that, sooner or later, this would hurt us. The real value of tests is known by those who write and sell tested products. Clients don’t know it. That’s why it’s essential to stand firm on your position, regardless of sales fears that, especially in this case, really don’t help.

We may still have doubts about how best to manage testing practices, but we have no doubt that the answer to the first question is YES: test, and talk about it with the client, bringing a big dose of common sense to the table and reflecting on what testing means in different situations.

To test or not to test? this is the problem