7c0h

Smartening up, Part II

In my previous entry about home automation I mentioned that I was ready to start plugging things into other things and connecting everything to a central computer running OpenHab. This is the story about how everything went wrong.

Networking

Things starting going badly from the get go when I discovered that whatever "vectoring" is it has rendered my DSL router obsolete. Only two (available) modems had the right technology: a TP-Link TD-W9960v and a Fritz!Box 7530 AX. I should have gone with TP-Link, but alas, I did not: I decided against the TP-Link because it was not supported by OpenWRT, but failed to notice that the other one was not supported either.

The reason I regret my choice is because the Fritz!Box router provides DSL, Telephone, Wifi, WLAN Mesh, Media server, NAS, some type of IoT compatibility, it updates automatically, and who knows what else out of the box. I have never owned any other device with such a large attack surface, and yet I am expected to put all of my data in it and then plug it to the internet? Yeah, no. I disabled everything instead, plugging all of my devices to my old router (which uses dd-wrt after OpenWrt dropped its support for it), relegated the Fritz!Box to a mere modem, and added every network access restriction I know to keep the later from entering my internal network.

Speaking of my old router, setting up sub-networks in dd-wrt was like pulling teeth thanks to my favorite internet problem: documentation that still refers to older, deprecated versions of software that nonetheless rank stupidly high in search engines because they have been around forever. In my case, all links pointed to this complicated method that has been improved by this much simpler method. Do you know how frustrating it is to debug network errors when bad configuration means you can no longer access the thing you are debugging? Because I do.

Smart lights

This is the only part that worked as intended, only to realize that what I thought I wanted is not what I actually needed.

I configured the lights to use their own Guest WiFi network that, once I'm done, will be disconnected from the internet entirely. That part worked well, and in fact I am typing way after midnight with the "candlelight" setting just because I can. The tricky part was the dilemma that came afterwards.

In order to work properly, the lights remain "on" all the time (to receive commands) and you turn them on/off via software. So what happens if I enter a room without my phone? At this moment I have three choices: * I can go back, pick my phone, unlock it, turn on the light, and then turn it off once I leave * I can turn the light off and then on again, turning the lights on. I then need to leave them on and turn them off via software once I have my phone back. * I can stay in the dark and convince myself that this is fine.

The solution to this problem is a small remote control I can keep on me all the time and turn lights on and off with as many programmable buttons as smart lights. I know this is a good solution because every single compatible remote is currently sold out.

Heating

Even before lights became an issue, my dream was to control my heating to turn it on when I get up and turn it off shortly after. My first step was to buy a cheap Bosch thermostat. This was a bad choice, as Bosch equipment require a central hub with constant internet connection. If you want to turn off your heating you have to send your request to Bosch headquarters, who in turn will tell your hub to tell your heating that it should turn off. Good thing I only bought one.

My second attempt was an Eurotronic Spirit Z-Wave plus. These devices connect via Z-Wave, a protocol that requires a dongle instead of internet connection. My research suggested an Aeotec Z-Stick, which would have been a fine choice if I had stuck to v5. Instead, I bought the newest v7 (newer is always better, right?) which has known compatibility issues and whose firmware can only be upgraded using Windows (a luxury I no longer have). So I bought a v5 and all those problems went away. Both the v5 stick and the Bosch thermostat will be landing on eBay eventually.

OpenHab

All of my equipment was supposed to be controlled with OpenHab, an experience that so far has been mixed: when it works it works great, but when it doesn't it is quite difficult to find out why. That compatibility problem I mentioned above regarding Z-Wave? All I found in the forums (before giving another stick a try) was the suggestion that my thermostat was asleep and that all I had to do was sit next to it and keep it awake for an indeterminate time until it worked. Once I changed the stick, however, everything worked just fine.

I installed OpenHab in my old laptop, where things worked mostly fine. But once I decided to move it to its own Raspberry Pi (using the OpenHabian distro) it all fell into pieces: my Pi is too underpowered, the memory card too slow, and the default Java installer doesn't support my architecture. So not only do I need a new dongle, but also a new Raspberry Pi.

Progress and next steps

The next steps are getting a proper Raspberry Pi, a remote control, and make everything talk to everything else. I still have some hopes for OpenHab - the thermostat have eventually been detected, the lights worked just fine, and the new old Z-Wave dongle does its job. I could look deeper into its voice recognition functions, but since my plan was installing Mycroft I might as well stick to that.

With the working up I have finally been able to re-plug my TV. The TV is old-ish and therefore controlled with a Raspberry Pi 3 running OSMC, with all of its content coming either from YouTube or the NAS (which I have once again turned on). This is not technically part of the smart home, but it is worth mentioning anyway. And no, I am not re-purposing this Raspberry Pi for OpenHab.

And last but not least, a word of caution: living in a non-functioning house (well, apartment) is exhausting, and I do not recommend following my steps until you know things will work fine.

I'll keep you updated.

How my brother's iCloud account was stolen

My brother got his iPhone stolen at gunpoint. This is the story of how he lost control of his iCloud account first (along with years of priceless memories, including my nephew's first steps) and how this couldn't have happened without Apple's incompetent support.

But before that, a plea for help: if you know of anyone who can help us get the account back (or rather, the priceless photos locked inside) please get in touch with the links at the end of the article.

Part 1: Locked out of iCloud

The first thing the thieves did (after running away, of course) was changing the phone number associated with the iCloud account. I do not know how they did this - it has been suggested that Apple will send a code to your phone, which the thieves obviously had. In any case, as soon as my brother tried to log into "Find my Phone" he was faced with a screen asking him to verify the phone number associated with the account, which was set to a number we do not recognize. It didn't matter that we still had the proper password for the iCloud account, nor that we still have control of the e-mail associated with the account. As far as Apple is concerned, if you don't know the phone number (which, again, the thieves changed) you cannot access your iCloud account. This is a known issue with iCloud security.

Next we got in touch with Apple, both via phone and Twitter. Phone support was useless, as all they would say was that they couldn't help us unless we knew the phone number. The closest I got to a victory was getting the phone representative to say out loud that she wanted me to provide the phone number the thieves gave, but that's about it. Even if we have the old phone number, the iCloud e-mail, the iCloud password, a police report and multiple IDs, Apple will not budge. Twitter support was even worse: after repeatedly asking them to read what I said two messages ago, I ended up getting this pointless, infuriating response:

We completely understand the concern with how important it is to have this resolved as soon as possible, and we were able to locate the cases from when you had previously reached out. Based on the information that you've provided, and the steps you and your brother have gone through, if your brother is unable to regain access to his Apple ID we would be unable to provide any additional methods to regain access to the account, and we would be unable to change the trusted phone number on the account. If you have any other questions or concerns regarding this issue, the best option would be to reach back out to our Apple ID experts at the link provided in our previous message (Note: that's the phone we called before).

Part 2: Losing iCloud for good

But the story gets even worse (better?). While we were stuck in phone support hell, my brother got a phishing SMS. He didn't recognize it as such, and lost the phone for good. The trick works like this: once you get a new SIM card (which the thieves can tell because the old one stops working) the thieves send you a phishing e-mail pretending to be from Apple. You follow the link, give your iCloud username and password, and now the thieves can unlock your iPhone and resell it. Crucially, this step only works because thieves know that Apple support will not help you: if Apple had been of ANY help then we would have recovered access long before the SMS and my brother wouldn't have followed the link.

According to Gizmodo (in Spanish) the next step would have been a phishing call with spoofed caller ID. But we will never know.

Sidenote: I reported both URLs (https://apple.iforgot-ip.info and https://apple.located-maps.info) to their hosting providers. Results have been mixed. PublicDomainRegistry.com (which belongs to Webhostbox) will not take down the hosting addresses unless they can see the phishing attack themselves (they won't check logs), but good luck getting a one-time link to work twice. UnifiedLayer.com was helpful, and I believe that GoDaddy revoked their domains.

Part 3: No more Apple

The iCloud website promises "all your stuff — photos, files, notes, and more — is safe, up to date, and available wherever you are". You have now seen what "safe" means in this context: it will be in the cloud, yes, but that doesn't mean you'll have access to it.

The question now is: would my brother choose Apple again? An iPhone is not cheap in general, and in Argentina less so. The current price for an iPhone 13 is ca. 400.000 ARS, which roughly translates to 2200 USD or 1300 USD at the unofficial rate (it's complicated). With an average monthly salary of 427 USD (according to Numbeo) you can see that getting a new iPhone is not a choice to take lightly.

What will my brother do? Paraphrasing from a conversation we had: "if I could get my account back I could consider getting a new iPhone. But if I have to start from scratch then it doesn't make sense. If I can't get my data back I'll probably get an Android phone instead".

Part 4: Conclusion and next steps

I have not entirely given up, but I'm not keeping my hopes up either. We are currently looking into whether my brother's wife can get access to his files (they had some kind of shared access), whether his iWatch can be of any use (it was logged into the iCloud account), whether small claims court is likely to help (I know it would work in the EU but Argentina is trickier), and whether anyone in my extended network can reach someone at Apple who is not an AI (thanks for nothing, LinkedIn).

As for next steps, I will be gifting my family access to some cloud storage, but unless I can get a service with competent tech support I may end up setting up a cloud of my own. Hopefully the loss of my nephew's first steps will not be in vain.

Do you have any ideas? Do you work for Apple? Then send me an e-mail or get in touch in Mastodon or even Twitter.

A bag for life

About 8 years ago I was searching for a new messenger bag - the one I had was good but too small, its only one big pocket was inconvenient, and the zipper was starting to fail. And since I was planning on bringing this bag with me everywhere, I needed a bag that could survive being dragged on a classroom floor, exposure to sun and rain, and fit under an airplane seat. Anything requiring "gentle care" was immediately out. Backpacks were also ruled out for a simple reason: in my country of origin backpacks are often cut open to steal their contents, so I wanted a bag I could always keep my eyes on.

My first purchase was a Timbuk2 bag which I bought on Amazon because there were no local retailers (looks exactly like this). Their marketing at the time promised "the last bag you'll ever buy" (or something to that effect), which was exactly what I wanted. But you know what the problem with lifetime guarantees is? That you can only judge them with customers who have used them for a lifetime, which disqualifies essentially everything made after at least the 2000s. Luckily (for you) you don't have to wait that long because a year or two after buying my bag the waterproofing started peeling off (along with a bit of the interior). As for the "lifetime guarantee", it turns out it is only valid for customers who bought their bag in an official shop. I got mine from Amazon, so no servicing for me. It has now been relegated to ugly sports bag status.

My second bag is an Aunts & Uncle's leather bag that I bought 7 years ago. The actual model is no longer for sale, but this one is very similar. I chose leather because, unlike more modern materials, I knew this would last forever (we still have bags from the 1800s around after all). And to be fair the bag's exterior is fine - it's a bit beaten up, but that's just part of its charm. The real problem is that the bag doesn't close anymore: the closing system uses Velcro (sorry, "hook and loop") which always made the bag impossibly loud to open. It was loud in daily life, it was loud in a quiet classroom, and it was definitely loud on a train at night. Eventually the material gave up and stopped closing altogether, which made it much quieter (good) at the cost of risking everything falling out (bad). I got in touch with the manufacturer after reading their sustainability compromise to get some proper belts installed, but the only solution they offered was to replace the Velcro and make it loud again.

I'm currently looking into leather working to see whether I could install some belts myself without making a mess. I would love to pay someone to do it, but apparently my city has exactly one woman who could do the job and she's on a sabbatical. I'll also probably peel all of the remaining waterproofing flakes of my other bag, spray it with some generic waterproofing, and make it look at least decent.

What would I recommend to someone who wants "a bag for life"? My suggestion would be to go for something simple that one can fix by themselves: while I currently don't have the leather working skills for doing the job, at least there's a clear path forward for my second bag. But sourcing, cutting and reapplying plastic liner for the first one? Good luck with that.

Further reading: some interesting comments regarding this article on the quality of modern home appliances.

ML models in Flask

Does this situation sound familiar to you?

  • You are a data scientist, you developed an ML model in Python (using PyTorch, TensorFlow, or something like that), and you'd like your users to interact with it,
  • You would like to make either an API or a web interface to your model,
  • Your model is big enough (and therefore slow) that you would prefer not to load it from scratch every time a user wants to use it, and
  • You know a thing or two about servers, but you don't have a deep background, you don't have the time and/or patience to get into it, you don't have the proper server administrator rights, or a combination of all three.

If that's your situation, this post is for you.

Flask is a Python package for the quick and easy creation of APIs that you can use to serve model predictions over the internet. And if your users need a GUI, Dash is a software package built on top of Flask that allows you to quickly create web interfaces - if you are familiar with R, Dash has been described as "ShinyApps for Python".

Typically you would use a "real" web server (Apache, NGINX, etc) to do the heavy lifting, but this post focuses on how to use Flask alone to quickly return results generated from an ML model. In particular, we will focus on how to keep a model in memory between calls so you don't need to restart your model at every turn.

WARNING: Flask is not designed to work this way. The Flask documentation itself tells you not to use their integrated web server for anything other than testing, and if you blindly expose this code to the internet things can get ugly. It will also be much slower than using a proper web server. And yet, I am painfully aware that sometimes you don't have the resources to do things right, and people telling you "there's a way but I'm not telling you because it's ugly" doesn't help. So remember: this solution is ideal for situations where you have low traffic, ideally inside an intranet and/or behind a firewall, and you don't have the technical help you'd need to do it right. But be aware of its limitations!

Method 1: global variables for simple models

Let's start with the simplest of web servers. This code exposes a single API endpoint /helloworld that receives a name and returns a greeting:

from flask import Flask
app = Flask(__name__)

@app.route('/helloworld/<name>')
def hello_world(name):
    return f'Hello, {name}'

if __name__ == '__main__':
    app.run(debug=True)

If you send a request to http://localhost:5000/helloworld/Test, you would get Hello, Test as a result.

Let's say that you now want to return a counter of how many times you have received a request - every time you get a request you simply increase a counter, and then you return that counter. One simple solution is using a global variable, like so:

from flask import Flask
counter = 0
app = Flask(__name__)

@app.route('/helloworld/<name>')
def hello_world(name):
    global counter
    counter += 1
    return f'Hello, {name}, you are request number {counter}'

if __name__ == '__main__':
    app.run(debug=True)

This code does work only in Linux (I think), but under some circumstances it could be all you need - all that would remain is for you to replace the variable initialization with code that loads your model into memory. You would use this method, for instance, when you need to perform a task with a long enough startup time, such as parsing a long list of JSON files. If that's your case, you can leave this server running like so:

  • Disable debug and open the server to the world by changing the app.run line to app.run(debug=False, host='0.0.0.0').
  • Install the screen utility (tmux is also good), start it typing simply screen in your console, run your server (python script_name.py) and leave the server running in the background (press Ctrl+A+D). The server will keep running until the computer is restarted.

Unfortunately for some of you, this solution doesn't work under Windows nor does it work if you use a "real" web server instead of the one provided with Flask. More important, it also tends to fail when using some ML libraries that are not happy with the parallel simultaneous access. If that's your case, your best solution is to create a sub-process (ugh) and communicate with your model via IPC (double ugh).

Method 2: Inter-Process Communication (IPC)

Before we jump into the code, we need to understand who is going to talk to whom and how. It goes as follows: the ML model will run in its own process, which we'll call the ML-process. This process can only be reached via a multiprocessing queue, a data structure where you put multiple elements which are later retrieved in the same order in which they were inserted and such that it can be shared with multiple processes.

Whenever you make a request to the API, Flask creates a new process that we will call a request-process. The first thing that this process does is to open a Pipe. You can think of a pipe like a special pair of telephones that can only talk to each other and where sound is only emitted when someone is listening - you can talk for hours into the receiver, but nothing will come out of the other end until someone listens (in which case they'll get all of your talking at once) or until the pipe is full. Whenever a request-process needs to perform a request to the ML-process, it does so as follows:

  • As we said above, the request-process opens a Pipe. It has two ends which we'll call the 'source' and 'target' ends. Remember, though, that despite their name communication can flow in both directions.
  • The request-process puts some data in the 'source' end of the pipe. Whenever someone picks up the 'target' end of the pipe they'll receive this data.
  • Next, the 'target' end of the pipe is put in the multiprocessing queue. If we stick to our analogy, it would be the equivalent of having two cell phones, putting one of them in a box, and mailing it to another person.
  • And now, we wait.

The ML-process is constantly monitoring the queue, and it will eventually receive the 'target' end of the pipe that we put in the queue. I say "eventually" because other processes are also trying to talk to the ML-process, and therefore every process has to wait for their turn. In our analogy, it is the equivalent of a person receiving package after package, each one containing a cell phone. Once the ML-process receives our 'target' end of the pipe it extracts the data, processes it, and puts the result back into the pipe using the 'target' end it received earlier. This result is then sent back via the pipe, where our request-process retrieves it and where it can be served back to the user that made the original request.

The following code does exactly that:

from flask import Flask
from multiprocessing import Process, SimpleQueue, Pipe

# This is the global queue that we use for communication
# between the ML-process and the request-processes
job_queue = SimpleQueue()
app = Flask(__name__)

# This is the process that will run the server
class MLProcess(Process):
    def init(self, queue):
        super(MLProcess, self).__init__()
        self.queue = queue
        # The slow initialization code should come here.
        # For this example, we just create a really bad cache
        self.cache = dict()

    def run(self):
        # Query the end of the pipe until we tell it to stop
        stop = False
        while not stop:
            # Receive the next message
            incoming = self.queue.get()
            if incoming == 'shutdown':
                # We got the magic value that tells us to stop.
                # Make sure this value doesn't happen by accident!
                stop = True
            else:
                # `incoming` is a pipe and therefore I can read from it
                data = incoming.recv()
                # Do something with the data. In this case, we simply
                # convert it to lower case and store it in the cache,
        # but you would probably call an ML model here
                if data not in self.cache:
                    self.cache[data] = data.lower()
                # Send the result back to the process that requested it
                incoming.send(self.cache[data])
        # If your model requires any shutdown code, you would place it here.
        pass

# This is a normal API endpoint that will communicate with the ML-process
@app.route('/helloworld/<name>')
def hello_world(name):
    # Create both ends of a pipe
    my_end, other_end = Pipe()
    # Send the data through my end
    my_end.send(name)
    # Send the other end of the pipe via the queue
    job_queue.put(other_end)
    # This process will now wait forever for a reply
    # to come via its own end of the pipe
    result = my_end.recv()
    # Return the result from the model
    return 'Hello, {}'.format(result)

if __name__ == '__main__':
    ml_process = MLProcess(job_queue)
    ml_process.start()
    app.run(debug=True)
    job_queue.put('shutdown')
    ml_process.join()

This code works well as long as there is perfect communication between all moving parts. If the ML-process hangs up, for instance, then no more data will be returned and all request-processes will keep waiting forever for a reply that will never come. The same will happen if you send the pipe to the server but you don't put any data in it. You can mitigate these problems by using the poll method of a Pipe (which looks whether there's any data and returns immediately), but you should be aware that synchronization errors are both common and mean to debug.

Note also that we have a special value that we use for instructing the ML-process to shut down - this is necessary to ensure that we clean up everything before exiting our program, but make sure no client can send this special value by accident!

Final thoughts

Is this a good idea? Probably not - if the developers of Flask themselves tell you not to do something, then "don't do it" sounds like solid advice. And we both know that there's nothing more permanent than a temporary solution.

Having said that, and as far as horrible hacks go, I like this one: if you are a data scientist then you are not here to learn how web servers work nor to have long discussions with your system administrator on what CGI means. You are here to get stuff done and getting your ML model in front of users as fast as possible is a great way to achieve that.

I used to know a way to extend this method to work with Apache (you know, a "real" web server) but I honestly can't remember it right now. If you need some help with that then reach out to me and I'll try to figure it out.

Smartening up, Part I

I always wanted a smart home. I don't have a particular use for it, I just think it's cool that I can yell at my living room and it will obey me. And since next month I'll be moving to an empty flat, it is truly now or never.

This is the first post detailing what I hope will be a painless experience and what I know will be a long list of frustrations. Today I will detail my general plan, and in future entries I'll let you know how it all goes.

My aspirations for this first stage are modest: I want to be able to control the lights in key spaces just by talking to a device. Not just turn them on and off, mind you, but also dimming the lights to certain levels. I also would like to have a smart mirror with my morning information (to-do and weather, mostly), but that is more of a stretch goal.

Starting from the top, I need smart light bulbs. Originally I planned on going with Ikea's TRÅDFRI light bulbs for their price, but I decided against them because they don't seem to play too well with open platforms and because they require an extra hub. I settled instead for the middle-priced Philips WiZ because they connect directly over WiFi (unlike it's expensive cousins from the Philips Hue line). I would have loved to use the cheap Hama smart lighting options, but my past experience with this company gives me little hope of their protocols being open or, for that matter, good.

Another factor in favor of WiZ was that they are supported by OpenHab thanks to the heroic work of one volunteer. Once it is properly configured I expect I'll be able to add complex commands like "dim the lights to 60% after 17:00 if it's winter" and stuff like that.

The voice commands will be handled by Mycroft, the privacy-focused alternative to Alexa and friends. I would really like to buy a Mark II, but given their delivery times I fear that I'll have to install my own version first (probably in my old notebook) and eventually migrate. Lucky for me, Mycroft and OpenHab are good friends.

The final part is networking. If you are familiar with my blog you may know how much I care for privacy, which is typically a problem when you want to install hardware that monitors your home 24/7. Therefore, all of the above-mentioned services will run in their own isolated LAN with no connection to the internet. Mycroft may get an exception depending on whether I would like to ask it about the weather, but everything else will stay isolated. This would also guarantee that I don't lose control of my lights when my internet is down. I have long ago flashed my WiFi router with dd-wrt which allows me to have multiple networks and define who can talk to whom.

Progress so far

Given that I already have the light bulbs, I tried to set them up using the Android WiZ app. This did not work: one key step of setting up the light bulbs is to register them on the cloud (for whatever reason), and the closed network made this impossible. I am fine in principle with the light bulbs phoning home once and then never again (combined with a VPN, the information they would expose would be minimal), but for that I would need internet and I still don't have any. I have also decided that two rooms will get "dumb" lights: the kitchen and the bathroom. These rooms are not "chill" rooms but rather "be there with a purpose and then leave" rooms, so there's no point in doing much with them.

And finally, one issue I have not yet decided is what to do about the microphone. Placing a Mycroft in the hallway would mean that I always need to yell at it, but I don't like the idea of my neighbors knowing that I turned on my lights at 3 AM and I doubt my neighbors would like it either. My best alternative so far is a small portable microphone - I read an interview sometime in the 90s about how Bill Gates' mansion was controlled with pins you were supposed to wear, and that seems reasonable enough to me. But I have yet to find something small enough.

Next steps

If I decide to go for the smart mirror, this guide seems like the way to go: I already have a Raspberry Pi I'm not using (used to be my NAS server) and an old laptop screen, and it's mirror film approavh is cheaper than those using two-way glass. The annoying part would be finding the appropriate control circuits for the screen, which is a can I've been kicking down the road for a couple years now.

I also would like Mycroft to play my music, but that would require me to install a NAS and set all networking correctly, which was not fun with all devices in the same network and will probably not be fun here.

I'll let you know how this all works out.