7c0h

Latest Post

Therac-25, LLMs and the Zen of Python

If you are a professional software developer you probably heard at some point the cautionary tale of the Therac-25, a radiation therapy machine that gave massive overdoses of radiation to at least six patients in the 80s and killed at least three.

There are several layers of problems explaining how all of this came to be, but today I'd like to focus on a specific thread. The (in)famous race condition in the Therac-25 was caused by two simultaneous bugs, but these bugs were not new: the software for the Therac-25 was ported from a previous version of the machine (the Therac-20) and those bugs were already present there. The key difference being that, while the Therac-20 had hardware interlocks preventing those faults, the Therac-25 engineers decided to implement them exclusively in software.

I'm digging this story up because today I read yet again someone suggesting that using ChatGPT to rewrite code from one programming language to another is a great idea. I disagree: a programming language is an opinionated way on how to better achieve a certain task and switching between world views without understanding how and why they do things the way they do is a recipe for inefficient code at best and weird bugs at worse. Simple example: if you're a Python programmer who came from C then you probably wrote loops like this:

# C programmer version
for i in range(len(mylist)):
    item = mylist[i]
    f(item)

# Pythonic code
for item in mylist:
    f(item)

This is a very simple example that highlights a fundamental misunderstanding of the language. The first code will run just fine, but it's generally not the proper way of writing Python code. For a more "real-life" example you can read the introduction to Learn Rust With Entirely Too Many Linked Lists where the author points out that writing linked lists in Rust is, from a Rust point of view, overall a bad idea and something to avoid entirely.

Real-case LLM

I decided to test my theory with Google's Gemini - I've seen students using it in their actual coding (probably because it's free) making it a fair choice. I asked the following:

Convert the following code from Python to Elixir:
def change_eight(my_arr):
    my_arr[8] = None

And the answer was:

def change_eight(my_arr) do
  if length(my_arr) >= 9 do
    List.replace_at(my_arr, 8, nil)
  else
    my_arr
  end
end
# This Elixir code ensures that the index 8 exists in the list before
# attempting to replace the value with nil. If the index is out of bounds,
# the original list is returned unchanged.

In my original code passing a list with less than 9 elements would throw an IndexError exception, which is consistent with The Zen of Python principle "Errors should never pass silently". But if my code relies on the exception to detect that a precondition is not being met, this rewrite has now ensured that the error will not be detected because the exception won't be triggered.

I added more context to the prompt, passing this code instead:

def change_eight(my_arr):
    my_arr[8] = None

if __name__ == '__main__':
    mylist = list(range(10))
    try:
        change_eight(mylist)
    except IndexError:
        print("Precondition failed")

    mylist = list(range(20))
    try:
        change_eight(mylist)
    except IndexError:
        print("Second precondition failed")

which lead to this rewrite:

def change_eight(my_arr) do
  if length(my_arr) >= 9 do
    List.replace_at(my_arr, 8, nil)
  else
    raise RuntimeError, "Precondition failed"
  end
end

if __MODULE__ == __ENV__["__MODULE__"] do
  mylist = Enum.to_list(0..9)
  try do
    change_eight(mylist)
  rescue RuntimeError -> error
    IO.puts(error.message)
  end

  mylist = Enum.to_list(0..19)
  try do
    change_eight(mylist)
  rescue RuntimeError -> error
    IO.puts(error.message)
  end
end
# This Elixir code is similar to the previous response, but it includes two
# test cases to demonstrate how the RuntimeError is raised in different scenarios.

The new version of the function is marginally better: Gemini now throws exceptions where expected, but it has also hard-coded the first error message - if the code fails on the second call the user will be shown an error corresponding to the first. I later asked for a C version instead of Elixir, at which point the error messages where gone entirely.

This is how critical bugs are born: when your software depends on (reasonable!) assumptions about the world and you send it through a pipeline (be it human or automated) that has no idea about what those preconditions are. And if you send your code through a system that's been designed to understand code exclusively at a syntactic level then you don't get to be surprised when the resulting code is not semantically equivalent to your original one.

Final words

Software development is not just about writing code. Software development is about understanding the environment in which your code runs and the decisions that lead to it - some of them reasonable ("this condition can never happen, the hardware will catch it"), some of them arbitrary ("let's write it in Perl"). The Therac-25 incident was made possible because someone decided to use code on an unfamiliar environment without considering the repercussions, the same way that Gemini did not consider "The Zen of Python" nor my error reporting strategy while rewriting my code.

There is more to software development than "data comes in, data comes out". Thinking about systems in terms of the context in which they run (software, hardware and social) is the best way to avoid finding yourself one day unpleasantly surprised.

Or, perhaps more relevant, unpleasantly shocked.

Further reading

If you haven't already, consider giving the classical paper "Four dark corners of Software Engineering" a try.

Older Posts

Calculator texts

Some weeks ago HN user wonger_ posted this list of words to Hacker News. This is a list of the words you can spell using only an upside-down calculator - the word "boobies" (5318008) is perhaps the most well-known, but it's far from the only one.

This comment by user chriscbr went a step further and annotated all words according to their Part-of-Speech while this one by user jprete raised the bar: can you do a long work of fiction only using calculator words?

I spent some time trying to make the longest possible text, and this post is a long, complicated way to say "probably not".

Randall Munroe of XKCD fame published some years ago this article on how to write texts using only a subset of letters - he was interested in phone keypads, but there's no reason why his code can't be adapted to our task. The basic idea is that of a trigram model: you train a model to predict "given these two words, this is an ordered list of the words that have the highest probability of coming next", you restrain your words to those that fit your constraint (in our case, letters that can be mapped to a calculator), train the model with some data, and you're done.

After downloading Randall's code, updating it for Python3 and making it slightly more efficient I trained it with as much Wikipedia text as I had patience for. The exercise gave some interesting word combinations, although nothing resembling a coherent long work of fiction:

  • Be less (would make a nice parody of "Be best")
  • Oh Ohio hills
  • High heel shoes
  • I'll go see his leg
  • His shoe size is big
  • He is obese she is his size
  • He is high, so is his boss, so is she

The next step was to use a more capable language model, and for that we move onto LLM territory. The idea is straightforward: during generation an LLM will look at the input words and make an ordered list of the most probable next words in the sequence. Usually we want to use one of the top words, but nothing stops us from using the most likely words that only uses a specific set of characters. We need to do this by hand because LLMs are incapable of backtracking - if they have generated "Hegel oozes ego" and then realize that they've written themselves into a corner, there's no way for them to backtrack and try something different. As a result sooner or later they all choose a word that doesn't fit the instructions, which is where we come into play.

Writing the prediction code was straight-forward, but punctuation was an issue: we do want to keep some punctuation for the text to look natural, but at the same time the phrase "Oil... Oil... Oil... Oil..." is more likely than something like "Hillbillies besiege his soil". And this meant that neither GPT-2 nor LLaMa 2 could generate anything long, although some short phrases ended up being interesting enough:

  • Bob is eligible.
  • He is 2 eggs high. See his gills.
  • He is 90. She... she is his lobolo.
  • Hello Bill, Bob is his big ol' goose.

At the end, the best approach for me was to go back to the beginning, use the annotated list of words and build phrases by hand. This led me to the short-story:

Bob sees Zoe boil his sole beige goose egg, sighs. She giggles.

HN user araes used ChatGPT and, after some sentences that didn't stick to the prompt (color me surprised) eventually generated the text:

Ellie sells loose shoes; Bill shills big goose oil.

Not one to be outdone, HN user bcherny used Claude 3.5 to obtain:

Ellie sees Bill. Bill sells bibles. Ellie lies, "Bill, sell bibles else." Bill sells bibles. Ellie gobbles bibles. Bill obsesses. Ellie sighs.

Which I now counteract with my own:

Leslie obliges. She'll go see Giles' bill "Oh, Ohio hills". She seizes Giles' high heel shoes, giggles. "Bozo, go sell blisses".

Or, if you want something even longer and feel like reading some really weird prose:

Giles sells big oil. Loose soil geologies is his hobbies, his Lego. He sees Shell be so big he is ill. He begs his boss Zoe, his gig is hell.

"Hell is high heel shoes. Hell is Boise, Ohio. Hell is hillbillies", his boss hisses. "Go be Bob's hell".

Giles obliges.

Bob oozes ego. Hegel is his Bible. Bob is so high he giggles. "Oh ho ho! Hello bozo".

Giles sighs. "Hello Bob. Zoe lobs blisses."

"Blisses? She lobs ills."

Can we do better? Probably. Could we generate "a long word of fiction"? I'm going to say "probably not": having 35 verbs and lacking both the letter "a" and the word "the" restricts the set of possibilities a lot. But given enough patience I bet you could easily write a page of more-or-less coherent text.

Let me close this post with some of my favorite generations that didn't make it into the longer stories because they are a bit inappropriate:

Repairing a shoe with a 3D printer

I got my 3D printing license last year, courtesy of the Stadtbibliothek Köln offering free printing to any card-carrying member who sits through their two-hours course "please don't burn down your fingers, the printer, or the library". The printer has been vital for replacing items that are too important to live without but too cheap to sell as spare parts, which brings me to today's topic: shoes.

If you haven't spent the last week obsessing over shoe spare parts you may not know that the heel cap is the part of the heel of a shoe that makes contact with the ground. A shoe without heel cap is useless because you can't tiptoe everywhere but there is no store around me that will sell a replacement for this specific model. A friend of mine lost one of those recently and you can imagine where this story goes from there.

The first step in making a replacement heel cap is to take measures of the part you want to replace. Since I didn't have my caliper at hand I took a picture of the heel where the replacement should go along with a reference of known size, a trick I picked from this scene of the 1999 movie "The bone collector". Here's the picture in all its grainy glory:

Picture of two shoes, one with missing heel cap and one with a 20 cent euro coin on top.

Next we need to trace the contour. A 20 cent Euro coin is 22.25mm wide so I imported the image in Inkscape, created a circle of exactly that size, and then resized the image until the coin in the picture matched the circle. I then traced the main forms on screen trying to take perspective deformation into account and ended up with a decent design.

Sketch of the contours of the main pieces that will be 3D printed.

While probably not the best tool for this specific use case, I used Blender to generate 3D shapes for these curves using extrusion, a process where you take (say) a circle, push it in a straight line, and end up with a 3D cylinder. I extruded every part individually, estimating 2mm for the bottom, 5mm for the guides, and 15mm for the main internal support. I finally saved the project as .stl for MakerBot Desktop, and we are ready to go.

3D rendering of the final piece.

One visit to the library and 30 minutes later, the first draft was ready for a test fit. I asked my friend for her shoe, took out my caliper for the necessary adjustments... and the draft piece fit perfectly on the first try. So all that remained was a second visit to make the piece tougher and deeper (just in case), and we are done.

Picture of the final piece.

This is not the first custom piece I print, but it's the one that made me realize how truly useful this technology has been to me. And you don't even need to know how to 3D model when sites like Thingyverse offer a wide array of ready-made designs for free. If you have the chance, you should definitely give it a try.

Building a Web App with nothing - Update

In my last post I mentioned a small webapp that I had built in an hour. This app was okay for what it was, but using it revealed that it needed a bit more love before it could be useful. And that's fine! After all, that was the whole point of making a quick mockup: to put it into the hand of users and see what their usage reveals about what's working and what isn't.

I have now written an updated version that you can see following this link. This new version has a couple small but important improvements:

  • It adds a second tab where you can see all of your data. Useful for knowing which expenses you've already added and/or to make sure that your records are saved properly.
  • It adds a button for clearing all data. I use this button not really to clear data, but rather to query the current date and time.
  • It now shows messages showing that data was inserted properly. As it turns out, not even the developer trusts a program that stores data silently and gives no acknowledgement whatsoever.

I still have not fixed the CSV export error. That one will have to wait for a future version.

Building a Web App with nothing

I recently had to solve a problem that I already faced before: I need an app to keep track of my daily expenses, it has to run on my phone (so I can enter expenses on the run before I forget), and pretty much nothing else.

Last time I scratched this itch I built an Android app that did exactly that, and it worked fine for what I wanted. But time has passed, I have forgotten almost everything I knew about the Android SDK, and I wanted to get this version of the app done from scratch in one hour - for a single CRUD application that doesn't do the UD part, this seemed more than reasonable. So this time I decided to go for good old HTML and plain JavaScript.

Building the interface

I decided to build the app in pure HTML - the requirements of my project are modest, and I didn't want to deal with setting up a server anyway (more on this later). I did want my app to look good or at least okay but, while I do have some art delusions, I was never particularly good at web design. Therefore, I am a fan of fast web design frameworks that take care of that for me, and this project is not the exception. I am weirdly fond of Pure.css but for this project I wanted something new and therefore I settled for good old Bootstrap.

If you have never used either of these frameworks, the concept is simple: you include their code in the header of your HTML file, you copy the template that better fits the look you're going for, and that's it. In my case that means that my empty template looked like this:

<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Spendings</title>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/css/bootstrap.min.css"
        rel="stylesheet" crossorigin="anonymous"
        integrity="sha384-T3c6CoIi6uLrA9TneNEoa7RxnatzjcDSCmG1MXxSR1GAsXEV/Dwwykc2MPK8M2HN">
</head>
<body>
    <main>
        <div class="my-4"></div>
        <div class="container">
            <h2>New expense</h2>
            <form id="main_form">
                <!-- Here comes the form fields -->
            </form>
        </div>
        <div class="my-4"></div>
        <div class="container">
            <h2>Export expenses</h1>
            <div class="row">
                <div class="col text-center">
                    <form>
                        <button class="btn btn-primary" onClick="export_csv(); return false;";>
                            Export data to .CSV
                        </button>
                    </form>
                </div>
            </div>
        </div>
    </main>
    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/js/bootstrap.bundle.min.js"
        integrity="sha384-C6RzsynM9kWDrMNeT87bh95OGNyZPhcTNXj1NW7RuBCsyN/o0jlpcV8Qyq46cDfL"
        crossorigin="anonymous"></script>
</body>
</html>

Once this was up and running it was time to add form elements. In my case that meant fields for the following information:

  • Expense: what it is that I actually bought.
  • Amount: how much I paid for it.
  • Date and Time: when did I buy it.
  • Category: under which category it should be stored.
  • Currency: in which currency is the expense - useful when I'm traveling back home.
  • Recurrent: this is a checkbox that I use for expenses that I'll have to re-enter every other month such as rent. I have not yet implemented this functionality, but at least I can already enter this into the database.
  • Two buttons: "Add expense" which would save this data, and "Export to CSV" to download the data saved so far.

The first three fields (expense, amount, and currency) look like this. Note that most of this code is a slightly edited version from the examples I copy-pasted from the official Bootstrap documentation:

<form id="main_form">
    <div class="mb-3">
        <label for="inputDesc" class="form-label" required>Description</label>
        <input type="text" class="form-control" id="inputDesc" required>
        <div class="invalid-feedback">Enter a description</div>
    </div>
    <div class="mb-3 row">
        <div class="col-sm-8 input-group" style="width: 66%;">
            <span class="input-group-text" id="descAmount">Amount</span>
            <input type="number" step=".01" class="form-control" id="inputAmount" required>
            <div class="invalid-feedback">Amount missing</div>
        </div>
        <div class="col-sm-4 input-group" style="width: 33%;">  
            <select class="form-select" id="inputCurrency">
                <option selected value="eur">EUR</option>
                <option value="usd">USD</option>
                <option value="ars">ARS</option>
            </select>
        </div>
    </div>
    <!-- A lot of other fields -->
    <div class="col text-center">
        <button type="submit" class="btn btn-primary"
        onClick="add_expense(event); return false;">
            Add expense
        </button>
    </div>

And with that we are done with the interface.

Storing the data

Web browsers have become stupidly powerful in the last years, meaning that they not only include a full programming language (JavaScript) but also at least three mechanisms for saving data in your device: Cookies, Web Storage, and IndexedDB.

This mostly works for my project: having a database in my phone saves me the trouble of having to set a database in a server somewhere, but there's still a risk of data loss if and when I clean my device's cache. The solution, as you may have seen above, is an "Export to CSV" button that I can use to save my data manually once in a while - I could have chosen to e-mail myself the data instead, but that's the kind of feature I'll implement once I have a need for it.

We now need to delve into JavaScript. Most of the code I ended up using came from this tutorial, with the only exception that my code doesn't really need a primary index so instead of writing:

const objectStore = db.createObjectStore("name", { keyPath: "myKey" });

I went for:

const objectStore = db.createObjectStore("name", { autoincrement: true });

Once you have implemented most of that code, you are ready to insert data into your database. In my case that's what the function add_expense is for and it looks like this:

function add_expense(event)
{
    // Validate the expense using the mechanism provided by Bootstrap
    // See https://getbootstrap.com/docs/5.3/forms/validation/
    form = document.getElementById("main_form");
    form.classList.add('was-validated');
    // These are the only two fields that need proper validation
    expense = document.getElementById("inputDesc").value;
    amount = document.getElementById("inputAmount").value;
    if (expense.length == 0 || amount.length == 0)
    {
        event.preventDefault();
        event.stopPropagation();
    }

    // Open a connection to the database
    const transaction = db.transaction(["spendings"], "readwrite");
    const objectStore = transaction.objectStore("spendings");
    // Collect the data from the form into a single record
    var data = {spending: expense,
                date: document.getElementById("inputDate").value,
                time: document.getElementById("inputTime").value,
                category: document.getElementById("inputCateg").value,
                amount: Number(amount),
                currency: document.getElementById("inputCurrency").value,
                recurrent: document.getElementById("checkRecurrent").checked};
    // Store this record in the database
    const request = objectStore.add(data)

    transaction.onerror = (sub_event) => {
        // This function is called if the insertion fails
        console.log("Error in inserting record!");
        alert("Could not save record");
        // Displays a (currently hidden) error message
        document.getElementById("warning_sign").style.display = 'block';
        event.preventDefault();
        event.stopPropagation();
    }
    transaction.oncomplete = (sub_event) => {
        // This function is called if the insertion succeeds
        console.log("Record added correctly");
        // Ensures that the hidden error message remains hidden, or hides it
        // if an insertion had previously failed and now succeeded
        document.getElementById("warning_sign").style.display = 'none';
    }
}

The function for creating the output CSV file is a simple combination of a call to retrieve every record in the database and this StackOverflow answer on how to generate a CSV file on the fly:

function export_csv(event)
{
    // Open a connection to the database
    const transaction = db.transaction(["spendings"], "readonly");
    const objectStore = transaction.objectStore("spendings");
    // Request all records
    const request = objectStore.getAll();
    transaction.oncomplete = (event) => {
        // Header for the CSV file
        let csvContent = "data:text/csv;charset=utf-8,";
        csvContent += "date,time,expense,amount,category,currency,recurrent\r\n";
        // Add every record in a comma-separated format.
        // Can you spot how many bugs this code fragment has?
        for (const row of request.result)
        {
            fields = [row['date'], row['time'], row['spending'],
                        row['amount'], row['category'],
                        row['currency'], row['recurrent']];
            csvContent += fields.join(",") + "\r\n";
        }
        var encodedUri = encodeURI(csvContent);
        window.open(encodedUri);
    }
}

The above-mentioned code has a fatal bug: I am building a CSV file by simply concatenating fields together with a "," in between, and that's likely to break under many circumstances: if my description has a "," in it, if my locale uses "," as the decimal separator, if my description has a newline character for whatever reason, and many more. I could solve this by escaping/replacing every "," in the input fields with something else, but let's instead learn the proper lesson here: do not build CSVs by hand in production!

Conclusion

And just like that we are done. I set out to get it done in an hour, and I almost made it - I had some issues with the database not initializing properly, which I ended up solving by bumping up the database version and forcing the database to rebuild itself. The glorious final version can be found following this link. This app saves all of your data in your local device, meaning that you can start using it right now. But you don't have to take my word for it. If you don't trust some random dude with a blog you can always check the source code yourself and make sure that the version I'm linking to is the same I described above. Isn't open source code great?

I can imagine a couple reasons why one would want to develop an app like this. Perhaps you need a working prototype that you want to put in the hands of alpha testers right now. Perhaps you want an app that has to work in exactly one device. Or maybe you have to develop in an environment where all you have is a text editor, a web browser, and nothing more. This would be unacceptable for a professional environment, but sometimes the problems you're trying to solve are so simple that scaling up in resources and costs doesn't make sense. Or maybe you would like to go down to the basics once in a while and realize that not every app needs React, a web server, and a cloud deployment.

I didn't manage to do all I set up to do. Some upgrades that I'm planning to add gradually are:

  • A fix to the CSV bug mentioned above.
  • A message letting you know that you expense was inserted successfully.
  • Implement the "Recuring expense" feature - I'm thinking of checking whenever you open the app whether the current month has any recurring expenses. If not, then the app asks you whether you want to insert them, one at a time (in case one of the expenses is no longer valid).
  • A hint of what the conversion rate for that day is, in case I don't want to save "I bought this in US Dollars" but rather "I bought this in US Dollars and that converts to this many Euros".

And finally, I should point out that this project was originally conceived as a proof of concept for streaming and coding at the same time. I have video of the whole experience but, given that the camera died on me halfway, you will have to rely on my word when I say that you aren't missing much.