Ξ

What is programming?

Published on 2017-11-05 code philosophy

The proper use of comments is to compensate for our failure to express ourself in code.
— Robert C. Martin — Clean Code

Once I was asked about my experience with test driven development (TDD) in a job interview. I told them that I was not really fond of it. This was not what they wanted to hear. I have never talked to anyone who had practiced TDD seriously. Still, it seems to be an ideal that many aspire to.

I personally feel that TDD is not compatible with my approach to programming. I will try to break down that approach by explaining what I don't like about TDD.

Design vs Production

For me, programming is about finding the best solution to a problem. It is not about writing code. These two approaches are often called "design" and "production".

"Producing" software means to take an existing spec and translating it into code. There may even be a fixed process in which this happens. Any concept that the programmers need to add to the spec in order to make it work are called "implementation detail". The main challenge here is to make everything work as expected and still keep it somewhat maintainable.

This is not what I do at all. I design software. People typically come to me with a rough idea of what they need and it is my job to understand their problem and come up with a solution. The main challenge here is to balance all different concerns and build a concise mental model from that.

I said that it is not about writing code, but code is still important. As any designer will tell you, you need to know your medium in order create good designs. The medium of programming is code. Knowing code enables me to build sound and well structured mental models.

What is really fascinating about code is that there is no further step of production needed: After building a mental model informed by code, you just need to write it down and are done.

If software is designed in this way there is no clear line between spec and implementation detail. The code is an immediate and concise representation of a mental model. It is its own spec and documentation all at once.

The process of design

It is hard to pin down how design is actually done. I guess I learned it through experience. Rich Hickey did a great explanation of some of the aspects in his talk Hammock Driven Development.

I am also very interested in Bret Victor's work on the topic:

Bret Victor — Learnable Programming

The first sentence captures what am I trying to say really well. But I strongly disagree with the second one: It claims that thoughts are somehow better if you can visualise them. This is really the same as claiming that movies are better than books. It desperately underestimates the power of pure thought.

For me, design is a rapid flow of coming up with new ideas, structuring them, and ruling out the unlikely ones. It really works best inside of my head. Any physical representation like writing or sketching mostly slows me down. The occasional scribble can be really helpful though.

Limitations

Sure, there are some limitations to this approach. You may not always be able to express your thoughts in code directly. Maybe the programming language can not easily express a concept. Or maybe the algorithm you need to implement is just complex and there is no way to make it expressive. In these cases just put that functionality in a function, give it a useful name (and maybe even a comment) and live with it.

The bigger concern however is that managers and graphic designers typically do not know code but still need to take part in the design process. The traditional approach is to add layers of abstraction. But these layers, apart from increasing overall complexity, ultimately form a wall between programmers and everyone else. Instead, I believe, we should encourage and empower people to understand the actual code in the first place, at least to some reasonable degree.

What is wrong with TDD?

at times I got sucked into that fundamentalist vortex, feeling bad about not following the true gospel. Then I'd try test-first for a few weeks, only to drop it again when it started hurting my designs.
David Heinemeier Hansson — TDD is dead. Long live testing

Test driven development is a design method where you first write tests, then do a simple implementation, and finally refactor the implementation. In TDD, the tests are the spec.

There are some practical issues with TDD that I am not going to get into. The talk Why TDD Is Crap by Eric Smith is an entertaining introduction to some more general criticism.

But from what I have written so far, you can probably guess that my main issue with TDD is something different. TDD is the very opposite of the approach I described above: While I prefer to take the medium of code into account when designing, in TDD design and implementation are strictly separated.

TDD adds a false sense of security: It may save me from implementation errors, but it will not save me from implementing the wrong thing. For this, I need to thoroughly understand the problem and the existing code. And as it turns out, having this understanding already prevents many implementation errors.

In practice, TDD also prevents large-scale refactoring. It assumes that your interfaces are stable (open/closed principle) and that it is possible to test the spec without also testing implementation detail. I believe that in practice, both assumptions are wrong. So while TDD encourages small refactoring that does not break any tests, it effectively blocks large scale refactoring that is essential to keep the system understandable.

Testing (and static typing, for that matter) trades conciseness for confidence. I totally agree that this is a good trade in some situations. But we need to acknowledge that it is actually distracting and harmful in many others. So while I think that testing can be beneficial, TDD is a different story.

Conclusion

You may have a completely different approach to programming. Maybe TDD gives you the structure you need to design and implement great software. Or maybe a visual representation of your code helps you to work on it. For me, I feel that these things are mostly distractions.

I still have hope though that TDD actually is the silver bullet as which it is described by its proponents. Maybe one day I will end up in a magical team that has perfected TDD and produces sublime software with it.

Until that day, I remain sceptical.