Should you talk to clients about testing?

This dilemma afflicts developers and salespeople when a new project has to be defined the budgeted for. The significance and likely consequences of the choice made also need to be explained to the client.

In this post I want to tell you about LeanPanda's experience of managing web development projects in which testing has been applied, over the past few years. I don't have any brilliant ideas, just some honest reflections and a few pieces of advice that are the fruit of past mistakes.

Whatever the size of the project that we're confronting, we always prioritize the "quality" of the work we do. This is a many faceted concept, taking in the quality of the management of the work, of life in the office, of aesthetics and of the development and of the code that we write. Testing — by which we mean a combination of Unit testing, Integration testing, TDD and BDD — is an indispensable part of the way put together quality code. Our use of these techniques is explained in a dedicated webpage.

This is a recurring doubt during our business meetings. Over time, we have experimented with different approaches, weighing up the pros and cons of:

NO: If we don't talk about testing with the client, the problem does not exist we run an insidious risk. When comparing our offer with the offers made by competitors who don't carry out testing, the client won't understand why we have allocated additional time and budget for it. In this scenario, even if a developer were, on their own initiative or out of necessity, to start writing tests, they would soon give up (disappointed and frustrated) due to a lack of time and resources.

So it's simple: if you don't talk about testing, it won't get done. Yo!

YES: Explaining what testing is, is genuinely difficult, and in 90% of cases the client won't have any idea what you're talking about. That said, a few technology enthusiasts will envisage hugely complex, fascinating scenarios that are far beyond the scope of what you think they require.

In both cases, during the detailed explanation, the face of our interlocutors will most probably fill with a mixture of skepticism, boredom, fear and — in a few cases — even a hint of disgust!

As developers, we are perfectly aware that everyone benefits from code that's been tested. The client will get what they want: a verifiable, functional product. The developer will sleep easier after deployment and have less fear of making potentially dangerous adjustments. In fact, the project as a whole will be able to benefit from having a long life built upon a solid foundation.

This is why we have decided to talk to our clients about testing.

We've had to ask ourselves how best to go about selling testing as part of a project's integrated development. After some trial and error, we've hit upon our formula for doing so.

EDUCATION: meetings with clients have become full immersion exercises in which we explain the agile and scrum approaches and go into the details of the method that we use. Testing forms a necessary part of this method.

QUALITY: we make it clear that, with regard to the infrastructure and code that we produce, quality is our strong point.

VIDEO: as a visual aid, we show our clients a video illustrating the complete process of a running test. We explain that where a human test requires an hour or two to carry out and has a higher margin of error, a testing suite can carry out the same work in a little over a minute, and guarantee that all buttons and forms are in correct working order.

FULLY TEST CODE: having 95% coverage means being able to insert new functionality without having to worry about regression, or interfering with the functionality of code that has already been written. It means having code that can be maintained over time.

Going ahead with a project without conducting tests will certainly lead to a faster initial development time and result in a lower initial cost. However, this approach can also lead to situations where trying to make modifications will present problems that are tremendously costly — if not impossible — to carry out.

A tested application will have a slower, and therefore more expensive, startup phase; however, this initial cost will be recovered over time, thanks to the fact that the addition of new functionality to the website will not create problems for what has already been assembled.

We know that code documentation is very important. We also know that it's often the first task to be abandoned as a project advances and the budget begins to run out. Carrying out testing is a perfect way of ensuring that a project is properly documented, as the tests themselves provide a description of the project's functionality.

Having a test suit helps when a new developer is added to the team, as it means that the new arrival won't be afraid to "get their hands dirty" by playing around with the existing code. It also makes it possible for the client to change supplier, which can happen for any one of a thousand reasons. It's also possible that the project will go well and the client will decide to set up an internal team for the maintenance and further development of the app. On that day, you will make a seriously good impression ;)

So then, why does this dilemma exist? It all seems very simple and logical... Where exactly is the problem?

In order to write the tests you need to have a clear idea of what you want the project to do. We know that clients, above all in the initial phase of a project, never know exactly what they want and are apt to change their minds rather frequently! It's important to remember that the problem here is not the testing itself; rather, it is the testing that shines light upon the real problems. The act of writing the tests itself makes it impossible to ignore problems and makes it necessary to confront and resolve them.

All developers need practice to get used to working with the idea that code has to pass through tests before further development can take place. In part, this depends on the skills and experience of the developer, but even a very good developer who isn't particularly confident with TDD will find getting used to this way of working and thinking extremely uncomfortable.

Writing good tests isn't easy. It requires testing something that does not yet exist, without falling into the trap of pre-engineering the application. There is also the lurking threat of over-engineering to be avoided: entering too deeply into the details of an element of functionality increases the risk of over-generalizing and implementing behaviour and tests that have not been requested and are not really necessary.

Then you get into the wonderful world of doubling up! In addition to the "visible" application, it's also necessary to maintain the test suit, which is itself made up of code! If the tests aren't treated with the same (or greater) care than the application code itself, then we risk putting our trust in a weak instrument.

Twice the amount of code. The ratio of test code to code is generally around 2:1. And if a test suite ends up having twice as much code in it as the app it's designed to test does, where is the greatest risk of an explosion of chaos occurring? Be very, very careful!

Twice the budget. We have to write 200% extra code, above all during the initial start-up phase. This takes time and, obviously, money!

You never stop learning from your mistakes. As I said, I don't have any brilliant responses to the dilemma posed at the beginning of this article. However, from the management point of view we have been able to come up with some ideas. We have found that there are two fundamental questions that clients need to answer with the greatest degree of transparency possible.

If the app is not going to require future development, it will automatically be released as legacy code. In this situation we will be able to say, with a certain sense of tranquillity, that tests won't be needed. If it's going to change "a bit", we will have to find out where and how much, so as to be able to identify the most delicate areas of the project.

This seems like an even simpler question! Everyone answers "extremely", no-one wants any bugs. But bugs do exist! And it only takes a little investigating to find areas of the application that are more sensitive than others, in which bugs are not at all acceptable. Let's imagine a situation in which we're dealing with a section of an app that deals with payments or the analysis of important health monitoring data. A bug here would be a much bigger problem than one on the "About us" page. By prioritizing bugs, we can gain a better understanding of what the most sensitive areas of a project are and identify the areas in which testing must, without doubt, be carried out.

If a project requires testing then it needs to have developers with testing experience working on it. It's quite likely that the results of the first attempts a developer makes at test writing will be unstable, and having a test suite in production that isn't solid is far from desirable.

The client will see the value of testing in the fact that their app won't break and will keep working solidly over time. That said, even if everything continues to work perfectly, clients can loose sight of the importance of testing and forget about it. For this reason, it can be a good idea to periodically forward reports generated by some testing tool that documents the code's state of health on to the client; the client probably won't understand a thing, but they will nonetheless be happy to know that their app is doing well!

Stay strong! The client will always want to save money, and sometimes it can be easy to fall into the temptation of encouraging them to make this apparent saving. However, it's important always to remember that, sooner or later, it'll come back to bite you. The real value of testing is known to those who write and sell tested products, not to clients. For this reason it's of fundamental importance to stick to your guns in the face of commercial fears which, in this case above all, will not help you.

We still sometimes have doubts about the best way of managing the practicalities of testing, but we do not doubt that the answer to the initial question of this article is "YES". Do tests, talk to clients about them, explain to them how much good sense testing makes and what its impact is likely to be in different situations.