[Off-Topic] Estimates Are Promises. Promises Must Be Kept.

Browsing Quora, I found a simple but interesting question: “Software Engineering: What is the hardest thing for a software engineer”. It’s a subject I’ve been reflecting on for many years, constantly. I answered in English but I think it’s important to also republish a Portuguese version here.

I’m a software engineer and what I see after 20 years of experience in different companies, markets, and teams is that practically every software engineer I’ve met starts with the premise that “code” is the goal and the solution to every problem.

The truth is that in the market in general (I’m not talking about exceptional cases of research and development or academia) software problems aren’t solved with software. This is the first and the hardest thing every software engineer struggles to understand and fights against.

Linda Vasquez: I know he made you a promise, but circumstances have changed.

Francis Underwood: The nature of promises, Linda, is that they remain immune to changing circumstances.

– House of Cards

One aspect I believe many cite as the hardest thing is “estimating,” because it’s so hard to arrive at correct estimates. And that’s the problem: the phrase itself is wrong and confusing.

Estimates, by definition, can never be “correct” — otherwise we’d say “predictions.” And it’s exactly because they’re two separate words. In a prediction the variables are determined and known, the path is predictable and automatic, it’s basically the lab scenario: “given the ideal conditions of temperature and pressure, in a vacuum, considering zero friction, a toy car moving at a constant speed of 10 meters per second, for 10 seconds, will travel 100 meters.” And estimates?

Outside the lab, we have estimates, and estimates are PROMISES.

Promises are made to be kept. A promise doesn’t resolve itself. You need to deliberately make an effort to reach it and keep your word. Likewise, an estimate is the same thing: you estimate and then you have to work hard to fulfill that promise.

“But the manager/client/investor/boss is constantly pressuring me to add more things, make changes all the time, breaking my concentration with irrelevant problems all day.”

Yes, they do this, and they’ll keep doing this, yesterday, today, and always. And the first thing most people do is become defensive, not wanting to commit, so they just decide not to give estimates or give estimates out of proportion to make sure any “extra” can fit. That isn’t a good default behavior.

If you make a promise to your daughter to be at her presentation at school, for example, and you fail to keep that promise, arriving 2 hours after it ended. That failure is yours and only yours. It wasn’t traffic, it wasn’t unexpected meetings at work, it wasn’t the weather or anything else. You made the promise, you didn’t fail to predict accidents along the way — you failed to preemptively leave earlier, get ahead of problems, and leave room for unexpected accidents. You left everything to the last hour and left at the last minute and, of course, shit happens.

Estimates are the same thing: you said a number, without commitment, without a sense of responsibility, and you did nothing to reach that goal beyond sitting down and writing code. You didn’t find explanations for the problems. And saying it once, your way, doesn’t imply that the other end understood — whoever initiates the communication has the responsibility to find the means for the final end to receive the message, otherwise communication didn’t happen. Communication isn’t saying, it’s being understood. And being understood is the responsibility of the one communicating.

So you failed at communicating, at managing your own time, at helping your peers with the problems. It was your responsibility — writing code was the smallest of the problems.

Having a sense of responsibility is exactly the second hardest thing every software engineer faces, because if the initial premise was that they were only responsible for “writing code,” they don’t see it as their failure not to have been understood in failed communication. So they never take responsibility for the failure, especially because it’s so much easier to blame everything else but themselves. And that’s precisely because the first hardest thing I mentioned above is the wrong premise that their only goal in life is to write code, that all problems are solved with code.

This isn’t true in software and isn’t true in many other areas, whether you’re a musician, a cook, a builder. Of course, as practicing professionals, you’re expected to be the best at your art. But that’s the bare minimum of the basic foundation, and there’s nothing exceptional about being technically good, no matter how impressive your technical capability may seem. It’s not a bit important that you can write the most absolutely perfect and elegant code if you’re writing software for the wrong problem. Or if you composed the most perfect Beethoven-style classical piece for a job that was a song for a children’s birthday party. Either you don’t accept jobs for kids’ parties, or you write the best pop music you can. If you stay, either you finish your job with exceptional quality, meeting expectations, or you accept the inability and walk away, not getting in the way of someone who can really solve the problem you couldn’t.

The only thing that anyone offering a service to other people needs to understand is that the goal is to implement the best possible solution, but for the correct set of problems. And the understanding that the hardest thing is precisely finding that correct set of problems. Clients come to us, professionals, precisely because they don’t know either. And as professionals it’s our responsibility to help find these problems, if it’s within our capability to deliver on a promise, and to keep that promise by managing whatever obstacle appears along the way.

It seems simple to say, but even the most experienced software engineers fail to understand this simple truth, and many manage to get by breaking promises using various tricks to hide the truth.

So learn the first and hardest truth of the world: Software is normally not solved with Software, it’s solved with Human capabilities. In a Pareto distribution I’d say 80% of every software problem is only solved when you invest those remaining 20% in Communication, Articulation, Clear and Rational Thinking, breaking Ambiguities, Negotiation, Commitment.

I’ll add by saying that it seems, then, easier to never make promises. It means never committing. It means never being a professional, because that’s exactly what differentiates an amateur or hobbyist from a professional: making promises and working to fulfill them. Another thing that does happen is you not fulfilling your side of the promise because it depended on the client, boss, etc. fulfilling their side of the promise first. A mutual exchange of promises is exactly the reason why there’s a referee called Legal Justice and the instrument that defines two promises has a name: it’s called a contract.

All this said, is it possible to keep all promises? No, unfortunately it isn’t. But it’s our responsibility as professionals to always be in pursuit of this ideal, not to create ways to avoid them.

I’ve talked more about the subject of what I believe a professional is in a 2011 post titled "[Off-Topic] Opinions, Truths, Democracy, and Ethics"