Category: Uncategorized

The latest rant: Dovecot (and a bit of ChatGPT, for good measure)

Earlier this week I was preparing a staging environment (on our live servers, so yes, that makes me TRWTF I guess) that required PHP 8.4. The servers were still running 8.2, but that upgrade was a bit overdue anyway, so I decided to run a dist-upgrade. After all, I’d been running 8.4 on dev for a while, so what could possibly go wrong?

Well, Dovecot could go wrong. I noticed the update, but it was from 2.3 to 2.4, so minor version, right? Not much could have changed, right? RIGHT?

Wrong.

This update completely (and I mean completely) overhauled the configuration syntax. Meaning that – essentially – our mail server was broken afterwards. Because fuck you backwards compatibility. Ugh. So I plunged into the official documentation. Which turned out to be not up-to-date. Or, ahead-of-date. Or something in-between. In short, not helpful.

After some head-scratching I decided I am one of the cool kids (or, at least, a cool middle aged man) and I would ask ChatGPT for help. Hey, what else could go wrong?

Oh boy.

I explicitly mentioned I was upgrading to Dovecot 2.4, which ChatGPT acknowledged was a breaking change (multiple times!). It then kept advising me <=2.3 legacy syntax. This went on, I kid you not, for about a day (not fulltime of course). Eventually, I managed to figure out (by myself) that the old %u, %n, %d etc. placeholders in SQL queries got superseded by %{user} and variants. Thank you Dovecot for not pointing that out to me in big, red letters. (And thank you ChatGPT for not telling me that until I figured it out by myself – “you’re absolutely right, these have been replaced by…” OF COURSE I’M FUCKING ABSOLUTELY RIGHT, I AM A HUMAN!!!1)

Anyway.

So I had that part working, but my mail users still couldn’t log in. Looking at the logs, the issue seemed to be that the MD5 “encryption” I’d been using for years was somehow deemed “insecure”.

Yes, I know MD5 is crappy encryption, but this was a mailserver with auto-generated passwords. Not a flying fuck was given. Except by Dovecot, which was suddenly flat-out refusing to authenticate, event though it fucking advertised supporting MD5 (yes, in doveadm output).

Despite my earlier dubious experiences, I decided to ask ChatGPT again, out of morbid curiosity. And it seems like it actually got it right this time: prohibiting MD5 seems to be a hard-coded limitation of Dovecot 2.4, despite it advertising it supports it. I poured another wine, lit another cigarette and started converting the most important mail accounts to SHA (the rest will complain later, I guessed).

All’s well that ends well? No, not quite. Though I could login to the server via IMAP now, and I could read my email, and send mail, mails weren’t being received by the server. So something else had changed.

Turns out (after a bit more head-scratching and ChatGPT “help”) that Dovecot now defaults to “stripping the domain name off of the receiving email address, since Linux systems default to usernames”. Yes it had a comment I should probably remove that if I was using something smarter. No, that’s not a helpful default. And it got introduced with no warning.

Tl;dr: upgrading to Dovecot 2.4 is a bumpy ride, at best. ChatGPT is not much help. And alcohol is an invaluable beverage.

Five years…

…stuck on my eyes, what a surprise, my brain hurts a lot, but that’s all we’ve got.

Yeah, it’s been a while, mostly because I’ve been working on legacy systems (hello Zend version 1!) that certainly had their share of wtfs, but weren’t that interesting to blog about.

However, a new rant is coming up! But first, I must purchase cancersticks (I still haven’t given up smoking, I AM vegetarian these days).

Stay tuned, kids!

Bootstrap: I fucking hate you

Okay, let me be slightly more nuanced about that statement: a few years ago Bootstrap had its merits (and I guess it still does). You can use it to quickly whip up a UI. I mean, it looks like every other UI you whipped up, but at least it isn’t totally horrible to look at.

Since I’m a programmer and not a designer, I thought that was handy.

But, Bootstrap has evolved, and so have I. What they offer these days is not only FUBAR, but it’s also dangerous. Let me explain.

I recently started working on a project that uses Bootstrap to da max. I never realised they took it that far (I mean, a class like d-none is handy for quickly hiding something if you’re not using AngularJS) but holy moly.

<div class="pb-0"> to set the padding-bottom to zero? Who on earth needs that? Can’t people write CSS anymore? But, more importantly, this is effectively the same as writing <div style="padding-bottom: 0"> (be it slightly shorter) and I thought we all agreed that that was a Bad Idea ™. You are now officially mixing markup with presentation, and the bleeding framework not only endorses but actively encourages it!

If that’s not bad enough, they offer stuff like d-none combined with d-lg-block. This means the element isn’t displayed, except on large screens. That’s fair enough, it’s no worse than adding your own @media rules to enforce this. What it does encourage, however, is idiots using this to add multiple blocks of more or less identical HTML – one geared towards mobile, the other towards desktop (or tablet). I mean, seriously, that’s not how it fucking works. Write your HTML once, and use CSS to tailor its appearance to different screen sizes.

I can only conclude one thing: Bootstrap has become a framework for amateurs that can’t be arsed to learn proper CSS. And that makes me sad.

Socket.io: ur doing it wrong

I hate it when documentation doesn’t advocate best practices. Even worse, that means that virtually every tutorial on the bleeding internet copypastas those same bad practices. I’m looking at you here, socket.io!

What’s the matter? All official examples – and, hence, the entire interwebz – use the following structure with anonymous functions:

io.on('connection', function (socket) {
    socket.on('some-event', function () {
        //...do something...
    });
});

Seems reasonable, right? I used this same approach in our dating-app, since that’s what everyone recommended. Who wouldn’t?

Well, woe on me.

A few days ago, as the app started gaining some traction, I started getting surprised by some weird out of memory errors on the webserver. “That’s odd”, I thought to myself, “there’s some traction but no way should it cause these errors just yet. Isn’t socket.io supposed to be, like, super-efficient?”

Well, yeah, but not when you’re Doing It Wrong.

See, using anonymous functions comes at a price: NodeJS creates a copy of them each time. In other words, each client was consuming memory for every event (and FlirtTracker has about 50 of them). So yeah, once I realised that the memory usage started making sense.

The solution? Use named functions declared just once (or, as I did ES6-style, anonymous functions assigned to a constant). Now NodeJS uses references to them and the whole memory problem goes away (well, for now, I guess it’ll pop up again when we really get some traction but it would be premature optimization to worry about that now).

Something that’s also not very obvious from the official docs (at least, I didn’t spot it) is that the socket events are actually bound to the current socket. In my original code I was passing the socket object around to utility methods that then returned the actual handler. Something like this:

module.exports = socket => (data, callback) => {
    // ...do something...
};

…and then in the main function run on connection:

socket.on('someEvent', require('./my-event')(socket));

But, because socket is bound, you can simply refer to it as this inside the handler:

socket.on('someEvent', function () {
    this.emit('someOtherEvent');
});

Actually, this solved a whole lot of other issues where I was passing all sorts of stuff around – I simply planted them on the socket in the main file, and they were now available via this! Add in some smart helper methods on the socket and I was able to reduce memory usage even more. For example, we also use a remote object pointing to a DNode server. I optimized that to this:

const getRemote = () => remote;
io.on('connection', function (socket) {
    socket.remote = getRemote;
});

(I used a method since I no longer trusted NodeJS to also use a reference of the object instead of a copy – I’m pretty sure Javascript passed objects as references like a good boy, but better safe than sorry.)

It even simplified our unit tests, for we could now use function.call or function.apply to bind a fake socket!

Last point to note: make sure the functions handling the events are actual functions, not arrow functions. I love arrow functions but one of their explicit features is they don’t have scope, so they can’t be bound to the socket. this will simply be an empty object in that case.

More fun with PHP!

We all know dates are hard, which is why we let libraries and built-in language features handle them. Right?

Well, turns out you can’t even trust those. I was just writing the following code, that needed to determine the first day of the next month for me. Using PHP’s strtotime function, this oughtta be easy, right?

$firstDateOfNextMonth = date('Y-m-01', strtotime('+1 month'));

Only, today is August 31st, so PHP actually turned this into 2017-10-01, because obviously that is correct. And yes, my timezone settings are local – var_dump(date('Y-m-d H:i:s')) gave me 2017-08-31 17:28:22.

For comparison, PostgreSQL does what I expect: 

# select NOW()+'1 month'::interval;
           ?column?            
-------------------------------
 2017-09-30 17:31:25.025405+02
(1 row)

Sighs time to file a bug I guess…

WTF PHP?

PHP has come a long way in recent years, and though it’s still not by far the most elegant language ever PHP5.3+ at least looks like an actual programming language.

But today I got bitten by something that totally flabbergasted me. Consider the following code:


<?php

$foo = new Foo;

Obviously, without any autoloading mechanism this gives a fatal error as class Foo is undefined. Now, modern versions of PHP have a handy feature where you can call ClassName::class and get the actual classname. This is handy for when it’s imported from some long namespace like so:


<?php

use Foo\Bar\Baz\Fizz\Buzz;

echo Buzz::class; // 'Foo\Bar\Baz\Fizz\Buzz'

So what do you expect happens when you do this to a non-existing class? After all, ::class is some sort of virtual constant on an actual class definition, right?

Wrong.

<?php

echo Foo::class; // 'Foo'
$foo = new Foo; // fatal error, class does not exist

Ehm…  That’s just… Wow.

The only reason I can think of why this works as it does is that the ::class is evaluated at compile time, while the actual instantiation is triggered at run time (thus causing autoloading to kick in).

To add to the party, I ran into this doing a Laravel project, and the autoloader swallows errors in the autoloaded class and simply says “sorry, no such class” (there was actually a typo in the loaded class, which I’d love to have been notified about before wasting an hour). I still need to find out what exactly is going on there, but I’m pretty sure it’s not Composer – I’ve never run into this behaviour on non-Laravel projects. (Also, the error occured in a Laravel “Command” which eagerly swallows errors too, because fuck you programmers debugging stuff.)

Tl;dr: just complain loudly if stuff fails. It’s fine. If I want to handle live errors more gracefully, I’ll just try/catch index.php or whatever myself based on some environment variable thank you very much.

Dating 3.0

Right. So, I don’t normally use this blog for promotion, but since I wrote about this app before and it’s now (more or less) done and live, I thought it deserved a mention. May I present to the interwebz: FlirtTracker.

The basic premise (which, credit where credit is due, my business partner came up with) is this: you’re doing something – hanging at a bar, chilling at a party, commuting, doing groceries – you name it – and you have a flirt with someone. Now, if you’re anything like me (and I’m guessing 99% of mankind is) you won’t be astute enough to ask for that person’s phone number. Instead, you let the moment pass and afterwards think WTFHOWCANISEETHEMAGAIN. That’s where this app comes in 🙂

At its core, you (discreetly) log your flirt in the app, and if the other person also does that, it suggests you as a match. So it’s fundamentally different from existing “dating apps”: the presumption is there already has been a real life spark. It’s got a bunch of other features and the website sports an explanatory video but in the stage we’re now in (soft launch and all) I’d mostly like to get feedback from real-life, out-in-the-wild users. (Yes we had a test panel but of course they won’t catch everything, plus we briefed them beforehand and I’d like to hear from “new users” as well.) So if you’d like to give it a spin, please do! Any feedback to support AT flirttracker.com will be highly appreciated – from “this don’t work” to “I have a better idea for feature X”.

The app is live and available for iOS and Android, but the web version is pretty similar in terms of functionality so anyone should be able to use it (and no we don’t support Windows Phone :P).

Tech details (I mention these partly to brag, but also b/c I’m expecting to need to hire people soon):

  • server backend partly PHP7, partly NodeJS
  • frontend AngularJS
  • app using Cordova
  • database = PostgreSQL

We plan to support WebRTC soon too as well, so I’m particularly interested in people with knowledge of that.

Got WebRTC working too today – we’re thoroughly testing it now since it’s still highly experimental, but looks good.

Of course, the app will work best if as many people as possible have it. We can’t find flirts if you’re the only one using it, so spread the word! There’s an “invite friends” option there, but just sharing this link is also <3. As is, it can’t be stressed enough, ANY feedback, even if it’s on the level of “this icon is 1px off” or “this totally sucks” ;).

Chinese takeaway

I don’t know what I accidentally dragged into the Firefox awesomebar to trigger this Google search, but I’m sure it’s pretty interesting…

google

Yes, that totally makes sense (not)

I’m developing a (rather cool, I think) website/app (it’s going to go live somewhere in the next few weeks, depending on how fast AAPL approves it) and as a part of that today I wanted to develop a circular countdown. A what? Well, essentially we needed a circular area to gradually ‘fill up’ clockwise until a certain time had been reached. In common speak, I was making a clock.

Back in the stone ages this would have involved 360 images each depicting the clock in a different ‘position’, but this is 2016 so I was sure I could do better.

I initially figured I’d use a canvas-element for this, but quickly found out I could do much better: SVG with a clipping filter. SVG allows you to define a clipping range with an arc. The syntax is pretty arcane (involving its own mini-language) but I managed to get it working quickly enough. It looks something like this:


<svg xmlns="http://www.w3.org/2000/svg" height="400" width="400">
<defs>
<clipPath id="wtf">
<path fill="#109618" stroke-width="1" stroke="#ffffff" d="M200 0 A 200 200 0 1 1 27 100 L 200 200 L 200 0"/>
</clipPath>
</defs>
<foreignObject height="400" width="400" clip-path="url(#wtf)">
<img width="400" height="400" src="/css/clock/light.svg">
</foreignObject>
</svg>

 

The exact syntax here isn’t important (Google it if you’re curious) but the M(ove), A(rc) and L(ine) specify commands, followed by a bunch of parameters. Yes it’s illegible, but it works. We then import an HTML image using <foreignObject> and specify the path we defined as a clip-path attribute (you’re going to understand why it’s called #wtf in a minute ;)). This simply refers to the ID we put on the clipPath SVG element, as the standard says “if you omit an actual URL, #id refers to something in the same document”.

(Actually, writing SVG in Angular involves a few more workarounds, but they’re not relevant for this problem. They were easily solved using mostly ng-attr- prefixes.)

So yay, I have this working. It looks roughly like this:

klok3

As you can imagine, the lighter area gradually progresses over time (in this random test case, 24 hours).

This is an Angular app, so I created a component just to display the clock so I could reuse it like a good programmer boy. That’s when things started getting weird…

My test case was working, and the directive was working fine where I used it first (which happened to be the app’s home page). I then wanted to use it somewhere else (e.g. /foo/) and the clipping filter (and any other filters for that matter) refused to apply. Okay, wtf?

I literally spent a few hours debugging this. I copied the SVG verbatim to my local testpage: it worked fine. Okay, so not an issue with the SVG being generated. Maybe some CSS causes it not to work? (I had a flashback to Explorer days of yore where some things would only work if an element had “display”, whatever that was. We just used to give it zoom: 1 to force IE to behave. If you have no idea what I’m talking about: consider yourself blessed.) Copied everything to an empty page on the same server, loaded the CSS and Javascript the actual site also loads: yup, still works. So no CSS issue.

Okay, next test. I actually whipped up a quick empty page in the actual application and tried to display the raw SVG (which I by now knew for a 100,000,000% certain was supposed to work). Now it fails again.

angry

Did I make an error copy/pasting somewhere? Nope, the SVG is exactly the same, yet it fails on one URL but not on the other.

Then finally I saw it. There was one other thing that differed between my raw HTML test page and the page generated from the application. The application has <base href="/"> in the head. And indeed, the ‘failing’ page was also attempting to load the home page in the background.

For those not familiar with Angular, it needs a <base> tag for routing to work. Normally not a big deal, but having this tag causes #id to no longer refer to the current document, but to the document specified in the tag. So my clipping directive was trying to load something from /index.html (where of course it couldn’t find it, since it was in /foo.html).

I’m sure there’s a reason this works like it does, since I could reproduce it in Firefox, Safari and Chrome. (At least they were consistent :)). But seriously, why? Okay, I get that <base> tells the browser to “consider every link in this page to be prepended with the href value unless declared more explicitly”, and from that perspective it sort of makes sense, but it kinda conflicts with the definition of “if omitted refers to an element in the current document”. I mean, what’s it gonna be people? The current document or whatever’s specified in <base>? Also, I tried working around this by using stuff like “url(./#wtf)” but that didn’t work.

Anywho, in the end I ended up just prefixing stuff with essentially window.location.href, and it worked everywhere. But I’m frankly surprised such a major pitfall isn’t documented better anywhere. Hope this saves someone a few gray hairs someday 🙂

Why I ffing *hate* mobile development

Okay, maybe not exactly *hate*, but it comes pretty darn close. I just need to rant about this, so stay with me.

I don’t often do mobile apps (due to the whole “stick with what you know” principle) but every so often I can’t avoid it and, being a web guy, I choose Cordova as my platform. I’m sure there are many advantages to building native apps, but I seriously don’t want to spend the time learning Objective-C, Swift or Android’s take on Java. Spare me. This last week was one of those times, so not having touched Cordova in a fair while I decided I’d give it a spin again.

First, the good news: the project has come a long way. Almost everything can be handled elegantly from the command line, which of course I as a Linux nerd highly applaud. Almost everything would “just work” out of the box, the plugin system has matured and the fact that you need AAPL hardware to even test an iPhone version isn’t their fault at all. So far so good.

So what I normally do is this: I start with laying out the basics in just a local HTML file. Firefox has a great “mobile view” built in, so you can quickly layout your app in general without going through tiresome compile/deploy cycles – just write and let Grunt live-reload your testing page. Sweet.

Of course after a while you need to work with something resembling an actual mobile environment, e.g. because your app is using Cordova plugins. Even sweeter: Firefox still has you covered! Recent versions come with a built-in WebIDE that offers – among a few other things I’ll probably never use – a simulator of a mobile device running FirefoxOS. I don’t think anybody actually uses FirefoxOS in the real world, but at least I have a simulator with familiar debugging tools at zero cost, and deployment is easy enough too (FirefoxOS apps are just zipped local sites, essentially). My versions were acting up a bit but that’s prolly due to the fact that I insist on running Debian Unstable on my desktop, and I managed to get it to work reliably by trying a few different versions.

But then I get to the “actual” version, which should run on iOS and Android (sorry WinPhone, Blackberry and whatnot lovers: this project doesn’t at the moment care about you ;)). Which brings me to reason I find mobile development excruciatingly frustrating:

If you want developers to love your platform, for Pete’s sake make sure your tools don’t suck.

I can’t stress this enough. Let me explain:

I started with Android, since at least I can run it on my Linux machine. The Android SDK comes with an emulator too. Ok, sweet. Only, it’s slower than a turtle with 3 broken legs and the fourth amputated. Seriously Google? This is a machine that happily runs multiple virtual instances of Windows next to each other, and your emulator is so slow it makes me want to chew my balls off? The package manager included seems to download a lot of updates too, but to be fair I hadn’t touched it in months so there will have been quite a few, and it did finish in under an hour. It’s not getting my speed-of-update prize, but it’s doable.

But then I turned to Apple. Oh boy.

Right off: people who know me will know I’m not a Mac fan, quite the contrary. Yes I own an iPhone and an iPad and a MiniMac, but that doesn’t mean I’m particularly fond of them. It just means I think they suck less than Android. In fact, if any carrier will support a FirefoxOS phone any time soon I’m game. But anyway, that’s no excuse to make your tools suck…

So I remote into my MiniMac via VNC. I used to use KRDC for that, but it was dropping the connection for no apparent reason so I switched to TightVNCViewer. Whatever, VNC is VNC. I could basically deploy to the iOS simulator using CLI tools (which was awesome since Xcode also makes me lose the will to live) so everything seemed fine. I had my iPhone hooked up the Mac for charging and was just about to deploy the test version to it, when I noticed there was a new iOS version out, so I figured I’d install that first while I was at it. Y’know, updates are there for a reason and it said “security”.

This was a big mistake.

After the upgrade had completed, the phone refused to charge via the Mac. I checked and it was charging fine on AC, so weird. But I had other stuff to do so I ignored it for a bit. Note: this was yesterday around noon. Around three, I did a bit of Googling and apparently making sure your Mac is also up to date could help solve this. It’s a device connected via USB just to power up. Seriously Apple, who cares about versions? But whatever, the Mac could use an update anyway. So I open the app store and check upgrades and sure enough, there’s some Xcode stuff and an OS update. Fine, I select “upgrade all” and go do something useful.

I got back from something else a few hours later, and the update seems to have finished. Nice. I connect the iPhone and I get a weird message about a developer disk image missing. WTF, I just updated, amirite???? A bit of Googling told me it was indeed a version mismatch (though why the message didn’t just say that and offer to download the correct version is anyone’s guess) so I check the appstore again. Oh, an upgrade for Xcode. Okay, let’s get it over with. Note: these upgrades are a few gigabytes each, and my MiniMac is on wifi. It’s not a particularly fast process. If you’re keeping count: this is the second upgrade to Xcode within 24 hours.

So I let it finish, hook up the phone, same error. I’m about to throw the entire machine through the window when I check the appstore again. Oh, now there’s another OSX update, because it’s obviously too much trouble to just package crap. Fine. I’m getting really annoyed by now, the update is another 7gig but apparently it needs to be done. Pet peeve: when it had finally downloaded and started installing, it warned that it might have to reboot a few times during the upgrade. What the f*ck do you mean “might”? You don’t know? Why the hell not?

Sooooo that eventually finished, I hook up my phone, and you’ve guessed it: same error. Streetwise by now I check the appstore and lo and behold: another XCode update! Of 2 frigging gigabytes. Apple, come on?

Well, nothing to be done, so I download and install it. After what seems like eternity it’s done, I restart XCode and… it needs to download required components! Why the fsck weren’t these in the download in the first place????

It’s now been chugging away at “installing documentation” (isn’t that on your, you know, website?) for over an hour. I think I’m just going to call it a day, but seriously: what the hell are these people thinking? I just want to deploy something to a perfectly fine device, why should the OS need to care about its version? And why, in 2016, can’t you use an efficient package manager like Linux has been doing for over a decade?

Argh.