Ξ

Difficult Problems and Hard Work

Published on 2022-06-25 code design

In “Difficult Problems and Hard Work”, David R. MacIver writes:

The distinction is roughly that something is hard work if you have to put a lot of time and effort into it and a difficult problems if you have to put a lot of skill or thinking into it. You can generally always succeed at something that is “merely” hard work if you can put in the time and effort, while your ability to solve a difficult problem is at least somewhat unpredictable.

I found that distinction interesting to reflect about my work, so I will try to add some of my thoughts to the discussion.

The problem with automation

My job consists mostly of identifying areas where people in the organisation have to do hard work and finding better ways to do that.

The first approach that most people will probably think about is automation. For example, instead of copying numbers from a piece of paper into a digital table, you could have an OCR pipeline that does that automatically. OCR of course is complicated technology. So in the context of this article, automation can be defined as “the process of replacing hard work with difficult problems”. And I am not entirely convinced that doing this conversion improves anything.

First, the automation needs maintenance, and that is still hard work. It may be less work than before, but you can not get rid of it 100%.

Second, you put the burden of dealing with complexity on the users. They were used to copying the numbers manually. Out of the blue, you come along with your fancy software and expect them to be thankful. But now they have to learn a new system where they have to scan the papers a certain way, trigger the pipeline somehow, and maybe deal with weird bugs or edge cases. This is all new to them and depending on how leaky your automation is you are asking a lot from your users.

Third, there is this weird encoding and decoding going on. First you transform hard work to a difficult problem. And how do you solve a difficult problem? By decomposing it into small steps – in other words, by turning it back to hard work. This mental gymnastics is easy for some people who are used to doing it, but I don’t think we should assume that everyone can follow along.

Fourth, hard work can trivially be split across a team. If there are 50 pages of numbers you can give 10 pages each to 5 people and get the work done 5x as fast. With difficult problems that is not as simple by a long shot.

Of course, none of that means that automation is bad. But it might not be the best choice in every case. While reflecting on this, I realize that I sometimes focus too much on eliminating hard work and do not actually consider the downsides of the alternatives.

A better option

Now you may be asking: “Why are there even numbers on paper?” And I think that is exactly the right question to ask here. Maybe we can just talk to the other department and ask them to send those tables by email instead of sending over printed pages. This would solve the issue without introducing any new automation!

In the beginning I wrote that my job mostly consists of identifying hard work and finding better ways to do that. The people who come to me mostly think about automation and are confused when I get back to them with a proposal to change their processes. And that is understandable. Having to see your work from a new perspective is not an easy thing to do.

In the end finding a better way is one thing, but communicating and implementing it is a difficult problem all by itself.