Why programming is hard

I have been giving programming languages a lot of thought recently. And it has ocurred to me that the reason why (reportedly) lots of people fail at learning how to program is because they are introduced to it at entirely the wrong level.

If you as a beginner search "Python tutorial" right now, you will get lots of very detailed, completely correct, very polished tutorials that will teach you how to program in Python, but that will not teach you how to program. Conversely, if you search for "how to program", the first results will be either completely useless advice such as "decide what you would like to do with your programming knowledge" or they ask you to choose a programming language. You might choose Python, in which case you are now back to square one.

One of the founding principles of my field is that "Computer Science is no more about computers than astronomy is about telescopes". In other words, programming is a skill that it's expressed typicall using programming languages, but it's not exclusively about them. And programming, the skill underlying all of these programming languages, is hard.

To be a good programmer, you need to master three related skills:

  1. Understand how to convert messy real-life problems into a clearer version with less ambiguities.
  2. Understand what the best practical approach to this problem is, and choose one as a possible solution.
  3. Understand how to use programming languages and data structures to implement that solution.

Mastering the first skill requires an analytical mind, and in particular forces you to see the world in a different way. If someone asks you for a program to keep track of how many people are inside a room, you need to stop thinking in terms of people and rooms and think in terms of numbers and averages. You also need to account for badly-defined situations: if a woman gives birth inside the room, is your solution good enough to increase the room count by one?

This part alone is quite hard. Some people make a living out of it as software requirements engineers, meeting with clients and discussing some approaches that would make sense. It also requires at least a surface level understanding of the type of solutions that one could realistically employ. If you ever wondered why your high-school math teacher always asked you to turn apples and trains into equations and solving for x, well, this is why: they were teaching you how to solve real-world problems with simpler methods.

In order to master the second skill "choose a viable solution", you need to read a lot about which problems are easy and which ones are hard. There are some problems that a programmer solves daily, and some problems for which the best known solution would still take thousands of years. If you think that finding new solutions to problems is interesting, I encourage you to go knock at the Math department of your nearest university. They do this for a living, and will be very excited to have you around.

Finally, the third and last skill "implement a solution" requires you to write it down in a way that computers can understand. Half the job requires understanding common concepts for structuring programs (variables, databases, data structures, networks, and so on), and the other half requires learning the syntax of your preferred programming language.

And here we reach the core of this post's answer. If you type "Python tutorial" right now, what you'll get are very detailed guides on how to acquire the second half of the third skill, also known as "the unimportant one". Sure, programmers love discussing which programming language is better and how not to write code, but here's a little secret: in the larger scale of things, it rarely matters. Some programming languages are better suited for specific tasks, true, but the best programming language is not going to be of any help if you don't know what you are trying to build.

At its core, programming is learning how to solve problems with a specific set of tools. And while you do need to understand how to use those tools, they are completely useless if no one explains to you how to solve problems with them. If knowing how to use a pen doesn't make you a writer, and knowing how to use a wrench doesn't make you a mechanic, teaching you a programming language and expecting you to become a programmer overnight is just going to leave you confused and frustrated. But remember: it's not you, it's them.

Appendix I: what does problem solving looks like?

Let's say someone asks me to "write a program to know who I have been in contact with in any given day", a problem known as contact tracing that has been in the news in the past weeks. How would the skills above come into play?

(Note: for the sake of simplicity, I am going to solve this problem badly. It's a toy example, so don't @ me!)

The first step is to model this situation in a formal, more structured way. Real people are difficult to work with, but luckily we don't care about most things that make them human - all we care about is where they have been at any point in time. Therefore, we replace those real people with "points", keep track of their GPS coordinates at all times, and throw all of their remaining attributes away.

We have now turned our problem into "tell me which GPS coordinates (i.e., points) have been close to my GPS coordinates at any given time". We can simplify the problem further by defining what "close to me" means, and we turn the problem into "give me a list of points that have been 1 meter or closer to me at any given time".

Next, we need to find a way to efficiently identify which points have been close enough to our coordinates. Since there is a lot of people in the world, we start by crudely removing all points that are more than 10km. away from me - this can be done very quickly, and it probably won't affect our results too badly.

We now need to refine our search, and therefore we take a dive into the geometry literature. After a quick look, I decided that building a Quadtree is the best solution for what I want to build. Note that I only have a passing knowledge of what Quadtrees are, but that's fine: once I have a hint of where the solution might be, I can search further and learn the details as I go.

And finally we get down to writing code. If our programming language doesn't already include an implementation of a Quadtree data structure, we might have to do it ourselves. If we choose Python, for instance, we need to understand how to create a class, how to use lists of objects, and all those other implementation details that our Python tutorial has taught us. Similarly, storing the list of points will probably require a database. Each database has a different strong point but, as I said earlier, knowing which database to use is not as important as knowing that some database is the right tool.

Let's now picture the same exercise in revers: imagine I come to you and say "Write me a program to know who I have been in contact with in any given day. Here's a guide on how to use Quadtrees in Python". You wouldn't find that last bit of any use, would you?