Yes, that totally makes sense (not)

I’m developing a (rather cool, I think) website/app (it’s going to go live somewhere in the next few weeks, depending on how fast AAPL approves it) and as a part of that today I wanted to develop a circular countdown. A what? Well, essentially we needed a circular area to gradually ‘fill up’ clockwise until a certain time had been reached. In common speak, I was making a clock.

Back in the stone ages this would have involved 360 images each depicting the clock in a different ‘position’, but this is 2016 so I was sure I could do better.

I initially figured I’d use a canvas-element for this, but quickly found out I could do much better: SVG with a clipping filter. SVG allows you to define a clipping range with an arc. The syntax is pretty arcane (involving its own mini-language) but I managed to get it working quickly enough. It looks something like this:


<svg xmlns="http://www.w3.org/2000/svg" height="400" width="400">
<defs>
<clipPath id="wtf">
<path fill="#109618" stroke-width="1" stroke="#ffffff" d="M200 0 A 200 200 0 1 1 27 100 L 200 200 L 200 0"/>
</clipPath>
</defs>
<foreignObject height="400" width="400" clip-path="url(#wtf)">
<img width="400" height="400" src="/css/clock/light.svg">
</foreignObject>
</svg>

 

The exact syntax here isn’t important (Google it if you’re curious) but the M(ove), A(rc) and L(ine) specify commands, followed by a bunch of parameters. Yes it’s illegible, but it works. We then import an HTML image using <foreignObject> and specify the path we defined as a clip-path attribute (you’re going to understand why it’s called #wtf in a minute ;)). This simply refers to the ID we put on the clipPath SVG element, as the standard says “if you omit an actual URL, #id refers to something in the same document”.

(Actually, writing SVG in Angular involves a few more workarounds, but they’re not relevant for this problem. They were easily solved using mostly ng-attr- prefixes.)

So yay, I have this working. It looks roughly like this:

klok3

As you can imagine, the lighter area gradually progresses over time (in this random test case, 24 hours).

This is an Angular app, so I created a component just to display the clock so I could reuse it like a good programmer boy. That’s when things started getting weird…

My test case was working, and the directive was working fine where I used it first (which happened to be the app’s home page). I then wanted to use it somewhere else (e.g. /foo/) and the clipping filter (and any other filters for that matter) refused to apply. Okay, wtf?

I literally spent a few hours debugging this. I copied the SVG verbatim to my local testpage: it worked fine. Okay, so not an issue with the SVG being generated. Maybe some CSS causes it not to work? (I had a flashback to Explorer days of yore where some things would only work if an element had “display”, whatever that was. We just used to give it zoom: 1 to force IE to behave. If you have no idea what I’m talking about: consider yourself blessed.) Copied everything to an empty page on the same server, loaded the CSS and Javascript the actual site also loads: yup, still works. So no CSS issue.

Okay, next test. I actually whipped up a quick empty page in the actual application and tried to display the raw SVG (which I by now knew for a 100,000,000% certain was supposed to work). Now it fails again.

angry

Did I make an error copy/pasting somewhere? Nope, the SVG is exactly the same, yet it fails on one URL but not on the other.

Then finally I saw it. There was one other thing that differed between my raw HTML test page and the page generated from the application. The application has <base href="/"> in the head. And indeed, the ‘failing’ page was also attempting to load the home page in the background.

For those not familiar with Angular, it needs a <base> tag for routing to work. Normally not a big deal, but having this tag causes #id to no longer refer to the current document, but to the document specified in the tag. So my clipping directive was trying to load something from /index.html (where of course it couldn’t find it, since it was in /foo.html).

I’m sure there’s a reason this works like it does, since I could reproduce it in Firefox, Safari and Chrome. (At least they were consistent :)). But seriously, why? Okay, I get that <base> tells the browser to “consider every link in this page to be prepended with the href value unless declared more explicitly”, and from that perspective it sort of makes sense, but it kinda conflicts with the definition of “if omitted refers to an element in the current document”. I mean, what’s it gonna be people? The current document or whatever’s specified in <base>? Also, I tried working around this by using stuff like “url(./#wtf)” but that didn’t work.

Anywho, in the end I ended up just prefixing stuff with essentially window.location.href, and it worked everywhere. But I’m frankly surprised such a major pitfall isn’t documented better anywhere. Hope this saves someone a few gray hairs someday 🙂

(Ab)using ngRoute to build a hybrid app

I love AngularJS. It makes building rich internet applications easy and does so in a structured manner. These days, whenever I need to fallback to jQuery (due to client requirements, for instance) it feels like going back to the Dark Ages. But enough praise.

I also love server side programming. Single page apps are fine and dandy and all, but there’s also a lot to be said for having a URI map to a specific blob of HTML consistently. It makes search engines happy (yes there are ways around that, but they’re clunky) and screen readers too. Besides, a simple wget should get the page I request, not some placeholder which is supposed to run heaps of Javascript before it renders in a usable way.

So for a while now I’ve been taking the following approach: web applications are written “old skool” in a server side language (PHP in my case, usually). The emitted HTML should return a usable page. Then, for all the interactive bits I use Angular directives. In short, I get the goodies from Angular but can still use “normal” routing via the server and it gracefully degrades if the Javascript barfs for whatever reason.

But, I was wondering, what if we can combine the two? What if the server would respond to a full page when a URI gets requested, but let Angular take it from there? Without, of course, having to duplicate our routing table? Then we can have the best of both worlds: a fluid experience for the user due to HTML5 push state URL handling, and HTML rendering logic on the server.

Turns out we can! And it’s actually a lot simpler than I thought.

Setting it up

First we need to include ngRoute so we can leverage Angular’s built-in URL handling. We could handle this manually via window.history.pushState, but why bother if the hard work has already been done.

let app = angular.module('foo', ['ngRoute']);
app.config(['$locationProvider', $locationProvider => {
    if (!!(window.history && window.history.pushState)) {
        $locationProvider.html5Mode(true);
    }
}])
app.config(['$routeProvider', $routeProvider => {
    // We don't actually _have_ routes, but just define these dummy routes
    // so the ngRoute logic will kick in and make our site HTML5 history
    // compatible.
    $routeProvider.when('/', {});
    $routeProvider.otherwise({});
}]);

The first config block conditionally sets Html5Mode for the $locationProvider as explained here. This is so that older browsers just fallback to “normal” routing, period. The post explains this further.

Next, we tell Angular that yes, / is handled by the $routeProvider, and we have no other routes. It’s imperative that you add both of these statements! If either is left out, ngRoute seems to give up. For this particular project, / will redirect to a language (/en/ for instance) anyway so it was perfectly safe. YMMV though – you might have to pick some dummy URL not otherwise used.

When we now reload the page all anchors will be “handled” by ngRoute, but won’t do anything yet. On to the next part!

Handling the route changes

ngRoute helpfully fires Angular events when it does something, so we’re going to tap into those:

let initial = true;
app.run(['$http', '$rootScope', '$cacheFactory', ($http, $rootScope, $cacheFactory) => {
    let cache = $cacheFactory('fooTemplate');
    $rootScope.$on('$routeChangeSuccess', (...args) => {
        if (initial) {
            initial = false;
            return;
        }
        let target = location.href.replace(location.origin, '');
        $http.get(target, cache).success(html => {
            $rootScope.$broadcast('fooTemplate', angular.element(html));
        });
    });
}]);

The exact naming doesn’t matter, but what happens here is:
1. When the $routeChangeSuccess event fires, we use the $http service to get the requested page from the server “under water”.
2. Once it returns, we broadcast an event with the HTML wrapped in a jqLite element as a parameter.
3. The calls are cached for efficiency.

Note that we also have an initial variable. This is because the route events also fire on page load, and since we already have our HTML we want to ignore the initial call.

Now, for each click you’ll see the page being requested in your browser console.

Changing the HTML

Define a directive (e.g. fooTemplate) that we’re going to apply to all elements that expect such a dynamic update:


app.directive('fooTemplate', ['$rootScope', '$compile', ($rootScope, $compile) => ({
    restrict: 'A',
    link: (scope, elem, attrs) => {
        $rootScope.$on('$routeChangeStart', () => {
            if (!initial) {
                elem.addClass('loading');
            }
        });
        $rootScope.$on('fooTemplate', (event, parsed) => {
            elem.removeClass('loading');
            angular.forEach(parsed, el => {
                if (!el.tagName) {
                   return;
                }
                // Replace & recompile the content if:
                // - the tagname matches
                // - the classnames match (if specified)
                // - the id matches (if specified)
                if (el.tagName.toLowerCase() != elem[0].tagName.toLowerCase()) {
                    return;
                }
                if (el.id && elem.attr('id') && el.id != elem.attr('id')) {
                    return;
                }
                let c1 = el.className.split(' ').sort((a, b) => a < b ? -1 : 1);
                let c2 = elem[0].className.split(' ').sort((a, b) => a < b ? -1 : 1);
                if (!angular.equals(c1, c2)) {
                    return;
                }
                elem.html(el.innerHTML);
                if (elem[0].tagName.toLowerCase() != 'title') {
                    $compile(elem.contents())(scope);
                }
            });
        });
    }
})]);

First we add a “loading” class to the element in question on $routeChangeStart. This is just for visual feedback to the user that something is loading under water, e.g. show a spinner. Again we only do that if we’re not in the intial run.

We then listen for our “template event” and traverse the returned HTML. If an element seems to match the element we put our foo-template directive on, we replace the HTML and – and this is important – run it through the $compile service. This allows any returned Angular components to also work after the page is updated.

Usage in HTML

Just apply the directive to any elements that contain “page specific content”:

<article id="content" foo-template>This is for a specific page and will be replaced when the URL changes.</article>

Todos/excercises

I’m going to tinker with this some more and might turn it into an actual package. In particular, it would need to:
– gracefully handle errors the $http call returns (e.g. a redirect to /login/ due to session expiration)
– do some intelligent pre-parsing on the returned HTML to it only contains elements actually tagged as templates (for efficiency)
– also handle form submissions in a likewise manner (unless of course Angular should handle them – probably check for the action attribute)
– clear the cache when needed, since POSTing stuff might invalidate certain HTML pages outright

Caveats

The HTML snippet the directive is applied to should be uniquely identifiable, so either give it an ID or make sure the combination of class/tagName is unique in the document.

Update

This is now an actual Angular module y’all can use: documentation GitHub Obviously directives have been renamed, but check out the documentation. It’s actually really easy to use (I’ve used it in a number of live projects already).

Websockets with Angular, socket.io and Apache

Seeing as I just spent the better part of the afternoon trying to figure this out, I thought I’d write a quick blog post on how I eventually got this working – both for other people struggling with this and for my own archive since I’ll probably forget again and want to look it up next time I do such a project 🙂

socket.io is a pretty awesome implementation of HTML5 websockets with transparent fallbacks (polyfills) for non-supporting browsers. However, its documentation can be…sketchy. For instance, their examples all assume everything is handled by nodeJS (including your HTML pages), which no doubt some people actually do but if you’re anything like me you’ll prefer to use Apache or nginx or the like in combination with a server side language like PHP for serving up your HTML pages. That’s not documented, ehm, extensively (to use an understatement), and also stuff like authentication isn’t really covered. So this is my solution, many thanks to Google, Stackoverflow and the rest of the interwebz:

1. The node server

socket.io is in the end a node module, so there’s no escaping that. Get it installed:

$ npm install --save socket.io http express

The most basic server script would look something like this:


let http = require('http');
let express = require('express');
let server = http.createServer(app);
let io = require('socket.io').listen(server);
io.on('connection', socket => {
    socket.on('someEvent', function () {
        socket.emit('anotherEvent', {msg: 'Hi!'});
    }
});
server.listen(8080);

Port 8080 is arbirtrary, anything not already in use will do. I usally go for something in a higher range, but this is good enough for the example.

To wrap your head around this: We have an HTTP server created using an Express app with socket.io plugged in, and it listens on port 8080. We can now run it using node path/to/server.js (in real life you’ll want to use something like PM2 for running it, but that’s not the issue at hand here).

2. The Angular client

There’s a bunch of Angular modules for tying sockets into the Angular “digest flow”. I chose this one: https://github.com/btford/angular-socket-io, but others should also work. The client script will be auto-hosted under /socket.io/socket.io.js (let’s ignore our port 8080 for now), so make sure that and the Angular module are loaded in your HTML (in that order, I might add). Adding an Angular factory for your socket is then as simple as


app.factory('Socket', ['socketFactory', socketFactory => {
    if (window.io) {
        let ioSocket = window.io.connect('', {query: ''});
        return socketFactory({ioSocket});
    }
    // I'm sure you can handle this more gracefully:
    return {};
}]);

This uses the defaults (the name Socket is arbitrary, you can call it Gregory for all I care), and sets up the query parameter. This is going to come in handy later on.

That’s really all there is to it; e.g. in your controller you can now do something like this:


export default class Controller {
    constructor(Socket) {
        Socket.on('anotherEvent', data => {
            console.log(data); // object{msg: 'hi'}
        });
    }
}

Controller.$inject = ['Socket'];

3. Configuring the actual web server

This is the part that mainly had me banging my head. Of course, we could just instruct socket.io to use our “special” port 8080, but that leads to all sorts of problems with proxies, company networks, crappy clients etc. We just want to tunnel everything through port 80 (or 443 for secure sites). An important thing to understand here: A web socket connection is just a regular HTTP connection, but upgraded. That means we can pipe it through regular HTTP ports, as long as the server behind it handles the upgrade.

Apache comes with two modules (well, as of v2.4, but it’s been around for a while) to handle this: mod_proxy and mod_proxy_wstunnel. “ws” stands for “Web Socket” of course, so yay! Only, the documentation didn’t go much farther than “you can turn this on if you need it”. In any case, make sure both modules are enabled.

This is the configuration that finally got it working for me (the actual rules vary slightly between socket.io versions, but this works for 1.3.6):


RewriteEngine On
RewriteCond %{REQUEST_URI}  ^/socket.io/1/websocket  [NC]
RewriteRule /(.*)           ws://localhost:8080/$1 [P,L]

ProxyPass        /socket.io http://localhost:8080/socket.io
ProxyPassReverse /socket.io http://localhost:8080/socket.io

A breakdown:

  1. First, we check if ‘websocket’ is in the request_uri. socket.io provides fallbacks like JSON polling for older clients – which is cool – and it does that by requesting different URLs and seeing which one works. I think in my version it tries /socket.io/1/xhr-polling next if websockets fail. Anyway, the point is: if Apache gets a request for the websocket URL, we rewrite to the ws:// scheme (on localhost, which is fine for now – if you’re running a gazillion apps this way you’ll probably want to handle this differently ;)) and just pass on the URL including query parameters (the P flag) and end it there (the L flag). If that works, the request was succesfull and our client support actual sockets. If not, the rewrite returns an invalid result and we move on to the next rule…
  2. For anything else under /socket.io, we now proxy to localhost:8080 via regular http and let the fallbacks handle it. This rule also makes sure that we can safely serve socket.io.js (since the URL doesn’t contain /websocket, it just gets forwarded).

If you’re using something else than Apache (e.g. nginx) there’s similar rules and rewrites available, but I’m not experienced enough in those to offer them here 🙂 The principle will be the same though.

4. Bonus: authentication

I rarely build apps without some form of authentication involved, and I’m guessing I’m not the only one. Regular authentication in a web app is usually something with cookies and sessions, but since socket.io is actually running on a different domain (localhost) – and besides doesn’t know about cookies – this won’t work. That’s where that ‘query’ option when we connected earlier comes in.

The query parameter is just something that gets appended to the socket.io requests as a regular GET parameter, and which can be read on the server. Exactly how you implement this is up to you, but a very simple (and by the way not extremely secure) option would be to just pass the session ID:


let ioSocket = window.io.connect('', {query: 'session=' + my_session_id});

And then in your node app on the server you can do something like this:


io.on('connection', socket => {
    socket.session = socket.handshake.query.session;
    // perform some validation, perhaps including a query to a database that stores sessions?
});

Server side languages like PHP by default store their sessions in proprietary flat files, so setting the server up to store them in a way that your node script can reach them depends on your platform. At least in PHP it’s pretty trivial to use a database like MySQL for that.

And that’s it really: you can now start building a real time application!

Graceful routing fallback in hybrid AngularJS apps

My current favourite way of working with the otherwise-awesome AngularJS framework is to only apply it to those sections of a page that actually need Angular-behaviour, and let common PHP solve the rest (what I call a “hybrid Angular app”, lacking a better term). While Angular does offer stuff like ngRoute and uiRouter to handle “HTML5-mode routing”, this comes with some other problems, the most notable being search engine indexing. While there are solutions for that they have their own drawbacks, so today I’d rather just have a regular site with some <element my-awesome-directive/> where needed.

I did however run into a problem: IE<10. (Yes, it’s Explorer again…) IE9 and before don’t support the HTML5 history API of course, so Angular has a fallback using hashed URLs. Fine. I assumed that as long as you’re not actually using routes, Angular would leave your URLs alone.

I was wrong.

For any regular URL /foo/, Angular insists on redirecting to /#/foo/ in older Explorers, which of course will just render the homepage with a useless hash URL since I’m not actually letting Angular handle the routing. Bummer. The solution turned out to be simple enough though: don’t force $location.html5Mode(true) if the browser doesn’t even support it. This will cause your IE<10 to just treat all URLs as regular links, will let Angular handle HTML5 URLs where you do define them (e.g. an image gallery) in supporting browsers, will still allow old IEs to run all other Angular code and thus provides the perfect graceful fallback.

Hence:

angular.module('mythingy', []).config(['$locationProvider', $locationProvider => {
    if (!!(window.history && window.history.pushState)) {
        $locationProvider.html5Mode(true);
    }
}]);