Off Topic: A Rant

Fair warning: this post is going to get a bit “philosophical” again. I don’t have the resources for a rigorous approach, with field research and statistical analysis of a huge sample. All I have is my own practical experience and whatever data is publicly available.

One such data point is this survey: Survey says 75% of IT professionals want to change jobs. It’s old, from the middle of last year, but I see it in practice. A few days ago some friends came to me with exactly this: “I’m unhappy with what I’m doing.”

These articles need to be taken with a grain of salt, since most of them report on the American market. This other one, for instance, says programmers are the highest paid in tech. In Brazil that’s not necessarily true. Here the programmer earns “enough,” and the bigger paychecks go to analysts (people with degrees in all sorts of fields who use technology as a work tool). And it’s precisely those analysts and consultants, some making up to R$100,000 a year, who complain the most about their own salaries.

Another complaint I hear all the time: “I’m not enjoying where I work anymore because it doesn’t ‘add’ anything to me. I want to move to other projects to learn more.” Wanting to change is admirable, but I believe most people are aiming at the wrong target. A colleague of ours put it well: “the vast majority of projects are all the same: data entry screens and more data entry screens.” That’s exactly it. It’s the descendant of the old field called Data Processing, which people now call Information Systems.

There should be more than that. Outside of programming you have Business Intelligence, Customer Relationship Management, Supply Chain, and several other cutting-edge areas, the ones that set apart the companies that already control the basics: accounting, sales, purchasing, manufacturing, maintenance, quality control.

In Brazil, the vast majority of “programmers” customize processes that already exist. One department wants to automate employee records: it buys or builds its own forms. Another wants a sales form that feeds the sales system and notifies an external partner to update inventory. Another needs screens to control a customer service workflow. Forms and more forms, always some variation of a form.

No wonder so many programmers are dissatisfied. Usually they can’t say why, but they feel something is wrong. And please, don’t take anything here literally. There are dozens of fascinating projects out there, with complex algorithms for medical technology, aerospace, engineering. I’m talking about the majority, not everyone.

This cycle of data entry is old. Since COBOL, since the 70s, data processing brought dynamism to the modern world, eliminating tons of paper and legions of bureaucrats whose only job was to fill out and control that paper. And it will never end, unless computers get smart enough to stop needing “mere” programmers. You never know.

Meanwhile, the market will keep needing “data entry programmers.” It doesn’t matter whether you use Struts, Spring, Rails, or Symfony: you’ll be building user records, client records, material records, content records, complaint records, contract records, address records, vendor records. The list is enormous. Especially where “modernization” hasn’t arrived yet, where bureaucrats abound and processes haven’t even been stabilized, let alone automated.

I started out building data entry screens in the late 80s, with dBase and Clipper. Most of the client-server era (2-tier) was exactly that: a database on the server and entry screens on the client. When the Web arrived, seductive applications showed up, search engines and webmail, and suddenly everyone wanted to build a website. Whoever invented TCP/IP, HTTP, HTML, SMTP, POP3 must have had a blast. For the rest of us, what was left was more data entry. An email screen is a form. An e-commerce screen is a form. Companies swapped their Visual Basic screens for browsers: data entry dressed up as web pages.

It’s no coincidence that so many frameworks keep popping up: Struts, Velocity, Lucene, Jackrabbit, Spring, Guice. A real programmer loves the plumbing. We like understanding how this mesh of code works, how to optimize it, how to make it stable, how to wire things together. Plenty of people in Brazil think this way, but far fewer than in the United States. We have nothing even close to a repository like Apache Jakarta. Brazilians contribute to various open source projects, but it’s a small fraction.

I can see it in the Rails case. Rails appeared recently, and it didn’t come out of the US. Even so, it spread fast over there. Here, aside from small communities like the competent RubyOnbr, we have nothing: nothing interesting in the media, nothing interesting at conferences (which are few to begin with). Nothing, zip, nada.

My book, “Repensando a Web com Rails,” barely sold its first print run of 1,000 copies. The excellent “Agile Web Development with Rails” sold tens of thousands abroad. Setting aside which book is “better” or “worse,” the meteoric sales, the dozens of other titles launched there, and the fact that mine is still the only one here are a thermometer of the cultural difference, in the market and among programmers alike.

Fair enough: we went through a dark period of closed markets and years of lag. But today I see no reason for programmers, especially the younger ones, to be so apathetic about technology.

At times like this I miss the late 80s (that’s the era I lived; older folks will feel the same about the late 70s). We were very few, and running into a programmer or even an enthusiast was rare. But we thought differently.

Even in the mid-90s, just before the internet boom, my friends and colleagues were all in love with technology. We talked about what was new, what each of us had learned, what we were testing. We traded ideas, argued techniques, and the goal was always to learn more. I knew people whose hobby was writing little animations in Assembly. Others bought electronic components to build their own MP3 player (iPods didn’t exist yet). Some spent hours building tools for this thing called the “internet” that was still new to us. Those were good times.

Then the Brazilian IT market exploded, for all sorts of reasons. Suddenly a lot of people saw the field as a quick way to make money. Suddenly universities were packed with kids learning tools instead of technology, graduating as information systems technicians where computer science graduates used to come out. Suddenly everyone “knew” Java. And finally the market was saturated with “programmers.”

Now all I hear from today’s “programmers” is “which language do you think I should learn that will make the most money?” You can imagine how frustrating that is for someone who already feels like part of the old guard. I never thought about the cost-benefit of learning time. I always took learning as a constant; if I made money along the way, great, but that was never the point.

Because of the current state of absurdity in the market, I witness things that scare me. At one client, they “integrated” the sales system with the activation system for a piece of equipment. The logic is simple: once the sale goes through, the customer’s equipment should be activated. Even without a two-phase commit solution (everyone knows what atomic transactions are, right?), the requirement called for safeguards to guarantee that.

Obviously there were no safeguards at all, and nobody ever asked: not the programmers, not the analysts, and certainly not the managers. The result: today several people are employed whose only job is formatting spreadsheets full of errors and fixing them by hand. More data entry screens were built to handle those errors. Did the project deliver automation gains? Sure, but far less than it should have. And nobody cares. I’ve seen worse.

It saddens me to see how little knowledge the bulk of programmers and the whole chain above them (programmers turned analysts, programmers turned managers) actually have, and how poor the architectures and solutions they produce are as a consequence.

Back to the questions people asked me: the unhappiness with what they do, the feeling of not living up to their “potential,” the urge to learn something new to earn more money. I have no answers for that because I can’t think that way. More than that: I believe those questions are fundamentally wrong. And a wrong question has no right answer.

I try to spread the culture I knew as a young programmer: learning. The excuses I hear are always the same: “I don’t have time to learn when I have to work to support myself”, “I can’t read English, so I need courses and can only learn one thing at a time”, “no point learning this ‘X’ thing because I don’t know if I’ll use it at work”, “I’m stressed enough at work; in my free time I just want to rest.” And so on. A long list of excuses.

It’s human nature to resist leaving your comfort zone and, worse, to complain when you become obsolete. “It’s the government’s fault”, “it’s capitalism’s fault”, “it’s my boss’s fault.” More excuses.

When I wrote my book, I did it for two reasons: to learn something new and to help others learn along. Learning and Teaching are the two fundamental motivators of every good programmer. I understand that in Brazilian reality, ignoring money is not an option. I found a balance between the two, and I don’t see why nobody else could. Standing still, sitting on the little you learned years ago, complaining nonstop, never got anyone anywhere.

If it’s worth anything, here are some personal suggestions:

• The more you learn, the better your solutions. Clients recognize a professional who knows what they’re talking about, and everyone can tell when someone is just bluffing.

• The demands that will grow the most are systems integration and analyzing large amounts of data. The only way to deal with that is knowing dozens of concepts and technologies. Learning one tool in a course won’t help.

• The technology cycle keeps getting shorter. A new version used to take years; now it takes months. By the time you finish the course, what you learned is already obsolete. Deal with it. Get ahead. Learn as much as you can before the technology is even available.

• Jumping from job to job doesn’t solve it. A new project won’t necessarily bring anything new (the odds of it being just another data entry screen are high). And leaving it to learn only when you already need to, inside the project, is too late: you’ll be overtaken by people who know what you should already know. A good programmer learns ahead of need. Once a poor-quality product hits production, the result is bad for the client and for your reputation.

• Try to innovate inside your own project. I’m not telling anyone to risk a project by bringing in things so new they could blow up the next day. There’s always a better way to do what you’re doing without touching the schedule or the cost. Learning to be efficient with quality is part of a programmer’s job, and it takes a lot of practice.

• Think you’re a good programmer but underused? Then put yourself to better use: create or join open source projects. That’s one of the qualities of open code: you get to use for free what others built, learn from a solid base, and still have the chance to give back.

• Stop complaining. Complaining builds a negative mindset, annoys everyone around you, and leads to no solution at all. A complaint is one more excuse, noise that hides the real problem. And the real problem is this: you complain about the job, the project, the boss, but the person you really want to complain about is yourself. Complaining is the externalization of the frustration of being who you are and failing to become more. And that is not a physical incapacity: it’s mental laziness.

• Forget courses and translated books: they’re always outdated. In the 80s we had no certifications or games of that sort. We slept with an algorithms book on the nightstand and woke up to a new language spec over breakfast.

Good cooks appreciate good food on vacation. Good engineers analyze their own car on a road trip. Good artists sketch new ideas on a restaurant napkin. Good professionals LOVE what they do, and not only at work. Loving it here means appreciating, respecting, and wanting to learn more about the profession, which is very different from taking work home. Da Vinci didn’t need courses. Mozart didn’t need courses. A course only helps someone who is already on track and knows exactly what they want out of it.

When I buy my iPod I want to know how it’s made (did you know it uses a hard drive with perpendicular bit recording, which increases data density?). When a spam filter blocks something, I want to know how it works (did you know some of the most efficient ones are based on Bayesian statistical algorithms?). When I install an operating system, I need to know exactly how it works (did you know your new hardware supports DEP protection, but it’s probably disabled on your Windows? By the way, has anyone stopped to find out what DEP is?). When I use someone else’s network, I want to be sure nobody will spy on what I’m doing (have you ever used an external SSH server to create dynamic tunnels?). When I buy a new TV, I want to get the most out of it (has anyone here bought a plasma TV and hooked an antenna up to it? Or worse: plugged the DVD player in with a composite cable and can’t figure out why the picture is so bad?).

And these things aren’t just bar-talk curiosities. I read a lot, nearly 100 feeds on the most diverse subjects. Over 100 posts an hour, every day. Plus dozens of podcasts and a bunch of books, printed or PDF. And I still think it’s not enough. A certification whose only merit is one more line on your resume is the same as nothing. I still stand by the principle that professionals who genuinely love what they do (and show it by learning more and more) will never be out of a job. The ones who start from the goal of making money, ironically, will have the hardest time getting it.