Slack's bait and switch

| categories: xmpp, foss, slack, converse.js | View Comments

Slack has finally decided to close down their IRC and XMPP gateways.

True to form, you can only read their announcement if you already have a Slack account and are logged in to a workspace.

Here's the gist of their announcement:

As Slack has evolved over the years, we've built features and capabilities —
like Shared Channels, Threads, and emoji reactions (to name a few) — that the
IRC and XMPP gateways aren't able to handle. Our priority is to provide a
secure and high-quality experience across all platforms, and so the time has
come to close the gateways.

They're of course being economical with the truth here.

Perhaps their XMPP gateway can't handle "Shared Channels" and "Threads", but that's because they purposefully stopped working on it.

A "Shared Channel" simply means a chatroom which people from outside your workspace can participate in. If a workspace is mapped to a members-only chatroom, then making something a shared channel simply means updating the members list or making the chatroom open (so anybody can join it).

Threads can be implemented by adding a <thread> element in the message stanza, as documented in XEP-201.

And emoji... there's nothing in XMPP that prevents people from sending emoji.


UPDATE: Several readers have pointed out that "emoji reactions" and emoji are different things. Emoji reactions can be tacked to a particular message.

Still, there's nothing fundamental about XMPP that prevents emoji reactions, and work is currently underway to add support for them.

The protocol is designed to be eXtensible (hence the X in XMPP) and new features are continuously being added.

Classic bait and switch

We all know the real reason Slack has closed off their gateways. Their business model dictates that they should.

Slack's business model is to record everything said in a workspace and then to sell you access to their record of your conversations.

They're a typical walled garden, information silo or Siren Server

So they have to close everything off, to make sure that people can't extract their conversations out of the silo.

We saw it with Google, who built Gtalk on XMPP and even federated with other XMPP servers, only to later stop federation and XMPP support in favour of trying to herd the digital cattle into the Google+ enclosure.

Facebook, who also built their chat app on XMPP at first allowed 3rd party XMPP clients to connect and then later dropped interoperability.

Twitter, although not using or supporting XMPP, had a vibrant 3rd party client ecosystem which they killed off once they felt big enough.

Slack, like so many others before them, pretend to care about interoperability, opening up just so slightly, so that they can lure in people with the promise of "openness", before eventually closing the gate once they've achieved sufficient size and lock-in.

On Federation

When we talk about "federation" in networks, we mean the ability to communicate between different service providers.

For example, email is federated. You can set up your own email server, and then send emails to people with their own email servers, or to people with Gmail or Yahoo! accounts.

You can email any other email address in the world, regardless of where that email address is hosted.

If email never existed, and a company like Slack today would come out with this brand new concept of "Electronic Mail", let's call it digimail, do you think they would standardise the digimail protocol and allow you to send messages to other digimail purveyors?

We all know the answer to that. They won't, and neither would Google, Microsoft or Facebook.

Heck, Facebook is actively trying to replace email since years.

The reason email is federated, is because it was developed before surveillance capitalism was a thing and because it was established and entrenched long before these companies came around.

There's a reason why your email address is still the de facto way to sign up for any service on the web (sometimes with one or two degrees of separation), and it's because of federation.

XMPP is designed to allow federation. Think about that. Instead of having to sign up to various different chat providers, all which try to lock you in and monetize your conversations, you could instead have one chat account, and use that to chat with anybody else, regardless of which chat provider they are using.

Alas, that's the dream, but because XMPP came much later to the scene, it didn't develop the critical mass as email has, and here we are. With dozens of chat apps, all non-interoperable and closed off.

What would it take for XMPP to take off?

One of the sad things that has come out of Slack's meteoric rise to success, has been how many free and open source projects have jumped over to using it (after previously using IRC or XMPP).

In so doing, they have closed off their discussions from search engines and they prevent people from accessing their past archives.

Slack has many cool features, and they work very well, I'm not going to deny it.

However, the XMPP Software Foundation has done a lot of work in recent years to enable protocol extensions that provide features that people have come to expect from chat applications, for example:

Unfortunately XMPP clients have been lagging far behind in various respects.

One of the main problems is funding. The modern digital economy is largely set up around surveillance capitalism and user lock-in.

So attempts to create software that doesn't follow these precepts, often end up unfunded or underfunded.

However, our "weakness", is also our strength.

XMPP clients, and the XMPP network can provide something that Slack never can. Federation, free and open software, interoperability, extensibility and user choice.


For the last few years I've been working in my spare time on making a JavaScript XMPP chat client, called converse.js.

Originally the idea was to make a Gtalk like chat client that you integrate in your website, and it can still be used like that.

A screenshot of converse.js as overlayed chatboxes

However, in the last year I've updated it further so that it can also be used as a fullscreen application, like Slack is used.

You can try the fullscreen version at

A screenshot of converse.js as a fullscreen application

If you have no-one to chat to, then come join the chat room.

This link will take you directly there (via

Converse.js still lacks lots of features that Slack has, but that's not because XMPP itself can't support those features.

What converse.js however does have, is that it's free and open source software, based on a standard protocol and it can be extended, updated and improved upon, by anyone.

We're actively working on adding new features and more and more people are joining in.

Moreover, anybody can host it and you can integrate it into any website.

Ultimately, I believe in the power and utility of interoperability and software freedom, even though the current trend is to close off and lock down.

These information silos are as powerful as we make them. If enough projects choose standardised protocols and FOSS software, we will be able to create viable alternatives that foster freedom instead of lock-in.


Hacker News discussion

This blog post triggered a lively discussion on Hacker news, you can read that here:

Read and Post Comments

Customizing and whitelisting SASL authentication mechanisms in Strophe.js

| categories: xmpp, strophe.js, foss, sasl | View Comments


If you've decided to read this fairly technical blogpost, then you probably have at least a rough idea what SASL is about and why one would want to create custom SASL auth mechanisms or whitelist the supported mechanisms.

I'll therefore provide just a very brief recap of the topics involved:

The Simple Authentication and Security Layer or SASL RFC 4422 is a framework for adding authentication support to connection-based protocols.

It provides an abstraction layer for authentication mechanisms, so that protocols, such as XMPP don't have to deal with the intricacies and complexities of supporting multiple authentication mechanisms.

It therefore makes auth mechanisms pluggable (if they are SASL compatible).

Strophe.js has supported SASL since a long time, but it didn't provide an easy way add custom SASL mechanisms, or to whitelist the mechanisms to be used.

Until now... or rather, since the 1.2.9 release.

Creating a custom SASL auth mechanism

To create a custom SASL authentication mechanism is fairly simple.

You can glean what's required by simply looking at how the default mechanisms are created.

See for example how the SASLPlain mechanism is defined.

And look as the SASLMechanism prototype to see the interface that the mechanism supports.

Perty much it boils down to creating constructor, settings its prototype to an invoked Strophe.SASLMechanism instance, providing its name, a boolean to indicate whether it should proactively respond without an initial server challenge, and an integer value specifying its priority amongst the supported mechanisms.

The default mechanisms and their respective priorities are:

  • EXTERNAL - 60
  • SCRAM-SHA1 - 40
  • DIGEST-MD5 - 30
  • PLAIN - 20
  • ANONYMOUS - 10

Then it's a matter of implementing onChallenge and any of the other methods provided by the SASLMechanism prototype.

onChallenge is called once the server challenges the client to authenticate itself or proactively if the mechanism requires that the client initiates authentication (configured with the isClientFirst parameter of Strophe.SASLMechanism).

So, lets create a fictional auth mechanism called SASL-FOO which works similarly to SASL-PLAIN, except that the password is encrypted with double-encoding ROT13 (hint: this is a joke).

We would then create the authentication mechanism like so:

Strophe.SASLFoo = function() {};
Strophe.SASLFoo.prototype = new Strophe.SASLMechanism("FOO", true, 60);

Strophe.SASLFoo.prototype.onChallenge = function(connection) {
    var auth_str = connection.authzid;
    auth_str = auth_str + "\u0000";
    auth_str = auth_str + connection.authcid;
    auth_str = auth_str + "\u0000";
    auth_str = auth_str + DoubleROT13(connection.pass);
    return utils.utf16to8(auth_str);

Whitelisting the supported SASL auth mechanisms

Now with SASL-FOO in hand, we can whitelist the supported authentication mechanisms by specifying a list of mechanisms in the options map passed in when we instantiate a new Strohe.Connection.

var service = '';
var options = {
    'mechanisms': [
var conn = new Strophe.Connection(service, options);

Bonus: Whitelisting SASL auth mechanisms in Converse.js

Due to the above changes it'll also be possible to whitelist SASL mechanisms in Converse.js (version 2.0.1 and upwards).

This is done via the connection_options configuration setting:


    connection_options: {
        'mechanisms': [
Read and Post Comments

Strophe.js and Converse.js now support passwordless login with client certificates

| categories: converse.js, xmpp, sasl, strophe.js, foss, openfire | View Comments


Did you know that x509 certificates, the certificates that webservers use to prove their identity during the establishment of an HTTPS connection, can also be used by a client (like your webbrowser) to prove its identity, and even to authenticate?

I'm talking here about so-called client certificate authentication.

Client certificate authentication is especially popular in environments with high security requirements. They can even be used to enforce 2-factor authentication, if in addition to a client certificate you also require a password. That usecase is however out of scope for this blog post.

With the release of Strophe.js 1.2.8, it's now possible to have passwordless login with TLS client certificates in Converse.js and any other Strophe.js-based webchat projects.

For Converse.js, you'll need at least version 2.0.0.

Here's what it looke like:

Logging in with an SSL client certificate

The technical details and background


The XMPP logo

XMPP supports authentication with client certificates, because it uses SASL (Simple Authentication and Security Layer).

SASL provides an abstraction that decouples authentication mechanisms from application protocols.

This means that XMPP developers don't need to know about the implementation details of any authentication mechanisms, as long as they conform to SASL.

Up til version 1.2.7, Strophe.js supported the SASL auth mechanisms: ANONYMOUS, OAUTHBEARER, SCRAM-SHA1, DIGEST-MD5 and PLAIN.

For client certificate auth, we need another SASL mechanism, namely EXTERNAL. What EXTERNAL means, is that authentication happens externally, outside of the protocol layer. And this is exactly what happens in the case of client certificates, where authentication happens not in the XMPP layer, but in the SSL/TLS layer.

Strophe.js version 1.2.8 now supports SASL-EXTERNAL, which is why client certificate authentication now also works.

How do you communicate with an XMPP server from a web-browser?

There are two ways that you can communicate with an XMPP server from a web-browser (e.g. from a webchat client such as Converse.js).

  1. You can use XMLHttpRequests and BOSH, which you can think of as an XMPP-over-HTTP specification.
  2. You can use websockets.

Both of these protocols, HTTP and websocket, have secure SSL-reliant versions (HTTPS and WSS), and therefore in both cases client certificate authentication should be possible, as long as the server requests a certificate from the client.

I'm going to focus on BOSH and HTTPS, since this was my usecase.

The HTTPS protocol makes provision for the case where the server might request a certificate from the client.


NOTE: Currently the only XMPP server that supports client certificate authentication with BOSH is Openfire, and funnily enough, only Openfire 3. In Openfire 4, they refactored the certificate handling code and broke client certificate authentication with BOSH. I've submitted a ticket for this to their tracker:

The authentication flow

So this is how the authentication flow works. I'll illustrate how the authentication flow works by using actual log output from converse.js


NOTE: My XMPP server's domain is called debian, because I was running it on a Debian server and because naming things is hard. In hindsight, this wasn't a good name since it might confuse the dear reader (that means you).

2016-09-15 12:07:05.481 converse-core.js:128 Status changed to: CONNECTING

Firstly, Converse.js sends out a BOSH stanza to the XMPP server debian, to establish a new BOSH session.

2016-09-15 12:07:05.482 converse-core.js:128
    <body rid="1421604076"
          to="debian" xml:lang="en" wait="60"
          hold="1" content="text/xml; charset=utf-8"
          ver="1.6" xmpp:version="1.0"
2016-09-15 12:07:06.040 bosh.js:749 XHR finished loading: POST "https://debian:7445/http-bind/"

The above stanza was sent as an XMLHttpRequest POST, and the above XML was sent as the Request Payload.

Strophe.js takes care of all this, so nothing to worry about, but sometimes digging through the internals is fun right? Right?!

2016-09-15 12:07:06.042 converse-core.js:128
    <body xmlns=""
          xmlns:stream="" from="debian"
          authid="fe0ee6ab" sid="fe0ee6ab" secure="true" requests="2"
          inactivity="30" polling="5" wait="60"
          hold="1" ack="1421604076" maxpause="300" ver="1.6">
            <mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl">
            <register xmlns=""/>
            <bind xmlns="urn:ietf:params:xml:ns:xmpp-bind"/>
            <session xmlns="urn:ietf:params:xml:ns:xmpp-session">

So now the XMPP server, debian, has responded, and it provides a list of SASL mechanisms that it supports. In this case it only supports EXTERNAL.

Luckily our webchat client supports SASL-EXTERNAL, so it responds in turn and asks to be authenticated.

2016-09-15 12:07:06.147 converse-core.js:128
    <body rid="1421604077" xmlns=""
        <auth xmlns="urn:ietf:params:xml:ns:xmpp-sasl"

Now here comes the tricky part. The XMPP server's BOSH servlet, asks the webbrowser (which is establishing the HTTPS connection on our behalf) to give it the client certificate for this user.

The webbrowser will now prompt the user to choose the right client certificate. Once this is done, the XMPP server authenticates the user based upon this certificate.

2016-09-15 12:07:06.177 bosh.js:749 XHR finished loading: POST

2016-09-15 12:07:06.180 converse-core.js:128
    <body xmlns="" ack="1421604077">
        <success xmlns="urn:ietf:params:xml:ns:xmpp-sasl"/>

The XMPP server responds with success and we're logged in!

How to set up client certificate authentication with Converse.js and OpenFire 3.10.3


NOTE: Thanks goes out to Dennis Shtemberg from Infusion, who initially tested client certificate authentication with BOSH on Openfire and on whose notes the following is based.

1. Install Openfire 3.10.3

The XMPP logo

On Debian(-based) Linux, you can simply do the following:

sudo dpkg -i openfire_3.10.3_all.deb

2. Configure Openfire's system properties

Open the admin console: http://localhost:9090/ (where localhost is the host the server is running on)

Navigate to Server > Server Manager > System Properties and add the following properties:

Property Value
xmpp.client.cert.policy needed
xmpp.client.certificate.accept-selfsigned true
xmpp.client.certificate.verify true
xmpp.client.certificate.verify.chain true
xmpp.client.certificate.verify.root true
sasl.mechs EXTERNAL

Make sure the xmpp.domain value is set to the correct host. If you're running Openfire on localhost, then you need to set it to localhost. If you're not using localhost, then replace all mention of localhost below with the xmpp.domain value.

3. Lay the groundwork for generating an SSL client certificate

First, make sure you have OpenSSL installed: aptitude install openssl Then create a directory for certificate files: mkdir ~/certs

Now create a config file called user01.cnf (~/certs/user01.cnf) with the following contents:

x509_extensions = v3_extensions
req_extensions = v3_extensions
distinguished_name = distinguished_name

extendedKeyUsage = clientAuth
keyUsage = digitalSignature,keyEncipherment
basicConstraints = CA:FALSE
subjectAltName = @subject_alternative_name

otherName.0 =;UTF8:user01@localhost

commonName = user01@localhost

The otherName.0 value under subject_alternative_name assigns the user's JID to an ASN.1 Object Identifier of "id-on-xmppAddr". The XMPP server will check this value to figure out what the JID is of the user who is trying to authenticate.

For more info on the id-on-xmppAddr attribute, read XEP-178.

4. Generate an SSL client certificate

  • Generate a self-signed, leaf SSL certificate, which will be used for client authentication.

    • Generate a private RSA key

      openssl genrsa -out user01.key 4096

    • Generate a sigining request:

      openssl req -key user01.key -new -out user01.req -config user01.cnf -extensions v3_extensions

      • when prompted for a DN enter: user01@localhost
    • Generate a certificate by signing user01.req

      openssl x509 -req -days 365 -in user01.req -signkey user01.key -out user01.crt -extfile user01.cnf -extensions v3_extensions

    • Generate PKCS12 formatted certificate file, containing the private key and the certificate. This will be the client certificate which you will log in with.

      openssl pkcs12 -export -inkey user01.key -in user01.crt -out user01.pfx -name user01

      • when prompted for export password enter: user01

5. Install the PKCS12 certificate on your local machine

Double click the pfx file and follow the steps to import it into your machine's keystore.

6. Import the x509 certificate into Openfire

sudo keytool -importcert -keystore /etc/openfire/security/truststore -alias user01 -file ~/certs/user01.crt sudo keytool -importcert -keystore /etc/openfire/security/client.truststore -alias user01 -file ~/certs/user01.crt sudo systemctl restart openfire


NOTE: The default keystore password is "changeit"

7. Create the user associated with the SSL client certificate

Go back to Openfire admin console, navigate to Users/Groups > Create New User and create a new user.

  • Username: user01
  • Password: user01 (This is not controlled by Openfire).
  • Click Create User

8. (When using Java 1.7) Patch Openfire

When trying to log in, I received the following error:

2016.09.08 00:28:20 org.jivesoftware.util.CertificateManager - Unkown exception while validating certificate chain: Index: 0, Size: 0

Turns out the likely cause for this is the fact that I was using the outdated Java version 1.7.

At the time, I didn't know that Java is the culprit, so I patched the following code

If you read the comments in the link above, you'll see there are two sections, with one being outcommented. I swopped out the two sections, and then recompiled Openfire.

After that, client certificate auth worked. The best way to avoid doing this is apparently to just use Java 1.8.

9. Test login with converse.js.

The converse.js logo

Now you're done with setting up Openfire and you can test logging in with Converse.js.

Download the latest version of Converse.js from the releases page.

To hide the password field (since the password won't be checked for anyway), you need to open index.html in your text editor and add authentication: 'external to the converse.initialize call.

Then open index.html in your browser.

In the converse.js login box, type the JID of the user, e.g. user01@localhost and click login.


NOTE: If things go wrong, pass debug: true to converse.initialize, then open your browser's developer console and check the output. Check especially the XHR calls to http-bind. Checking the output in the Network tab can also be very helpful. There you'll see what Openfire responds to requests to its BOSH URL.


Client certificate authentication is a bit of a niche requirement, doing so with BOSH/HTTP even more so.

However, I expect webchat XMPP clients to become more and more prevalent in the coming years, even on the desktop, for example when packaged with Github's Electron (an Electron version of converse.js is planned BTW, based on the fullscreen version inverse.js).

The fact that this works because of SASL-EXTERNAL authentication being added to Strophe.js means that this functionality is not only possible in Converse.js, but all webchat clients built on Strophe.js (granted that they use version 1.2.8 or higher).

Unfortunately XMPP server support is lacking, with only Openfire supporting this usecase currently, and not yet (at the time of writing) in the 4.0.x branch. To see whether this gets fixed, keep an eye on the relevant ticket

Read and Post Comments

Open Source software and the expection of free labor

| categories: foss, open-source, economics | View Comments

Over the last few years of starting and then maintaining an open source project that has received a decent amount of attention, converse.js, I've noticed some interesting things about the expectations some people have towards developers who work on FOSS (free and open source software).

Predictably irrational

Book cover: Predictably irrational

People of course love to receive something for nothing. Dan Ariely, in his book "Predictably Irrational" illustrates some of the biases people have when it comes to free stuff. When confronted with the words "free" (as in gratis), people do things that are irrational and are at odds with how a rational actor (the mythical homo economicus) is expected to behave, which is the bedrock upon which most economic theories are based.

The outcome of the various studies Ariely conducted was consistent: when faced with multiple choices, the free option was commonly chosen. With the opportunity to receive something for free, the actual value of the product or service is no longer considered. [1]

“Most transactions have an upside and a downside, but when something is FREE! we forget the downside. FREE! gives us such an emotional charge that we perceive what is being offered as immensely more valuable than it really is.”

Dan Ariely

The biases regarding "FREE!" apply not only to monetary costs, but also to time. We forgo some of our time when we wait in line for free popcorn or to enter a museum on a free-entrance day. We could have been doing something else at that time, so there's a resultant opportunity cost. [1]

Freedom isn't free, it costs folks like you and me


These biases of course can also come into play when people evaluate free (as in beer) software. In the same way that people didn't take into consideration the cost of the time they spend in trying to get something for "free", people often also don't consider the non-monetary costs of using FOSS.

A common retort that usually surfaces on Slashdot, Reddit or Hacker News whenever a discussion around using a Linux distribution on the Desktop takes place, is “Linux is only free if you don't value your time”.

That's of course completely true. I do value my time, took that into consideration and still concluded that I want to use GNU/Linux and free and open source software.

Using FOSS requires a certain amount of commitment, and it should be clear to the user why they are willing to go that route (freedom from vendor-locking, the ability to control and keep private your data, the ability to modify the code to your liking etc.).

I think people have been hyping the "FREE!" aspect of FOSS way too much.

Software for nothing and your support for free




I consider a certain amount of support and maintenance as a requirement for a successful open source project and not something you (as the developer) can ignore.

I try to channel bug reports and feature requests to the Github issue tracker and general support questions to a mailing list, where hopefully other people would also be willing to share the load by answering questions.

So while I complain about people wanting "something for nothing" below, I invariably mean people who write to me directly, instead of on the issue tracker and who are often trying to get me to work on something right away.

So, when considering that many people don't properly evaluate the costs involved in using FOSS, some requests and emails that I sometimes receive start to make sense.

“Please guide me”

One common recurrence, is to be contacted by someone who is integrating converse.js into a project for a paying client, and somehow got stuck. Perhaps they didn't read the docs or perhaps they don't have the requisite technical skills to do the job. These emails sometimes have a pleading, desparate tone to them. Perhaps to instill some sense of guilt or obligation or perhaps just because the person is really desperate and under time pressure.

What gets me every time however, is that as far as I can tell, these are people working for commercial businesses who get paid for the work they do. They then trawl the web looking for hapless FOSS developers to do their work for them for free, or as expressed in the commonly used phrase in these kinds of emails: “Please guide me”.

The novelty and warm fuzzy feeling of altruistically helping strangers solve their problems disappears like mist before the sun when you realise that they're getting paid for the work you're doing for them right now.

And make no mistake about it, maintenance and support for an open source project is work and sometimes even drudgery. The fun part is writing new code or trying out new things, not helping people who can't be bothered to study the documentation.

We need a feature and we hope you'll do it for free

Another common theme is emails where people somehow just assume that I'll implement some feature for them. At first this presumptuousness startled me.

I think it's totally fair to ask when the project is charitable and the people involved don't receive any payment themselves, but that's often not the case.

Instead, the underlying assumption appears to be that I love working on open source projects so much that I'll do it all for free and that I don't have ideas on what to work on next.

Sometimes people qualify their requests by stating that they're a small non-profit. Non-profits do however pay out salaries, don't they?

I'd be willing to reduce my hourly rate when working for a non-profit with a good cause, but I'm most likely not going to do work for free.

The software is free, but the time spent working on it costs money

A nuance that's perhaps lost on many people, is that I have often worked on converse.js for money. There was a rather long "bootstrapping" phase in the beginning where the project wasn't good enough for anyone to actually use or pay money for further development, but after the project stabilized I started getting small paying gigs of custom development on converse.js.

In all cases I made it clear that the Mozilla Public License forces me to open source any changes I made to the covered files, and therefore the work I did for these paying customers (bless their hearts) was open sourced as well.

The point is that while the software is free (as in beer and as in speech), the time spent working on it costs money.

Either someone else pays me to spend my time working on it, or I end up paying by doing something for free while I might be getting paid doing something else (opportunity costs) or by taking time away from other activities.

FOSS development costs money, either the developer is commissioned, or they pay for it themselves (perhaps unwittingly).

Doing work for free devalues it and takes the piss out of actual paying customers

The last point I'd like to make, is that by taking on these requests to do free work for commercial entities (and non-profits), I'm not only devaluing my work, but I'm also disincentivising paying customers (which includes non-profits).

After all, why would anyone pay me to do anything if I'm so eager to please that I'll do it all for free?

The only reason I could see to do that, is to get that mythical "exposure" that's often also sold to web and graphic designers.

The Oatmeal comic: "Exposure"


So what do you do if you need work done and can't pay for it?

Free and open source software is a beautiful, world-changing and paradigm shifting idea. However, software developers, like all people, need to be paid for their work, also when they work on FOSS.

If you can't pay for software development, then you can still try to incentivise FOSS developers in other ways, but be aware that it'll be more difficult.

One important lesson that I'm glad I learned early in life, is that when you're asking someone to do something for you, then you need to explain to them why it's in their best interests to do so.

People inherently look out for themselves. It's perfectly natural and doesn't necessarily mean they're selfish to the point of being anti-social, it just means that they need to take care of themselves and that they can't expect other people to do it for them or to even have their best interests at heart.

So when desiring something from someone, such as their help, the best approach is to explain to them what's in it for them

For example, if you want a friend to help you out with something, let's say to join a beach cleanup project, you don't tell them why it'll be good for you, you explain to them that it'll be an opportunity to chat, to meet new people, to go for a swim and to have the enjoyment of a clean unspoiled beach.

This is simple stuff, but many people apparently don't know this.

So if you want someone to help you with a software project, explain to them why it would be in their best interest. If you can't find a reason why, then perhaps it's actually not in their best interest and you need to create an incentive for them.

Money works pretty well as an incentive, but there are other ways as well. One sure-fire way to build up goodwill and gratitude (that might translate into more help and assistance) is to contribute. If you can't write code, fix typos in the docs, evangelize the project or contribute in other areas which you have some expertise, like translations, design, UX, helpful feedback etc.

FOSS development is a community effort and a team sport. There'll always be people who try to take more than they give, but on average, humans are matchers. When given something, they want to reciprocate and give back. Keep that in mind when you're trying to get something for nothing.


[1](1, 2) Wikipedia article on Predictable Irrational
[2]Sung to the tune of Freedom isn't free
[3]Sung to the tune of "Money for nothing" by Dire Straits.
[4]From The Oatmeal
Read and Post Comments

Sprint Report: Merging Mockup and Patternslib

| categories: mockup, javascript, patternslib, austria, sprint, foss, plone | View Comments

Alpine City Sprint, 21 to 26 January 2015

This is a report on what I did at the Plone Alpine City Sprint organized by Jens Klein and Christina Baumgartner in Innsbruck between 21 and 26 January 2015.

Firstly, I want to thank to them for organizing and hosting a great sprint and for being such friendly and generous hosts.

My goal for this sprint was to work on merging the Patternslib and Mockup Javascript frameworks and I'm happy to report that very good progress was made.

Long story short, the merge was successful and it's now possible to use patterns written for either project within a single framework.

Before I expand on how this was achieved during this sprint, I'll first provide some necessary background information to place things into context and to explain why this work was necessary.

What is Patternslib?

Patternslib brings webdesign and development together.

Patternslib's goal is to allow website designers to build rich interactive prototypes without having to write any Javascript. Instead the designer simply writes normal HTML and then adds dynamism and interactivity by adding special HTML classes and data-attributes to selected elements.

For example, here is a Pattern which injects a list of recent blog posts into a element in this page:

Click here to show recent blog posts

The declarative markup for this pattern looks as follows:

<section id="alpine-blog-injected">
    <a href="#portlet-recent-blogs"
       data-pat-inject="target: #alpine-blog-injected">

        Click here to show recent blog posts

The dynamic behavior comes from the fact that I have the Patternslib Javascript library loaded in this page.

On pageload, Patternslib scans the DOM looking for elements which declare any of the already registered patterns. If it finds such an element, it invokes a Javacript module associated with that pattern.

In the above example, the <a> element declares that it wants the Inject pattern applied to it, by virtue of it having the pat-inject HTML class.

Patterns are configured with specific HTML5 data properties. In the example above, it is the data-pat-inject property which specifies that the target for injection is the #alpine-blog-injected element in the current page and the content being injected is specified by the href attribute of the <a> element. In this case, the href points to an anchor inside the current page, but it might just as well be a link to another page which will then get injected via Ajax.

More information on how to configure pat-inject can be found on the patternslib website.

Each pattern has a corresponding Javascript module which is often just a wrapper around an existing Javascript library (such as Parsley, Select2 or Pickadate) that properly defines the configuration options for the pattern and properly invokes or applies the 3rd party library.

So, in order to do all this, Patternslib can be broken down into the folowing components:

  • A registry which lists all the available patterns.
  • A scanner which scans the DOM to identify declared patterns.
  • A parser which parses matched DOM elements for configuration settings.
  • The individual Javascript modules which implement the different patterns.

What is Mockup?

Mockup, declarative interaction patterns for Plone..

Mockup was inspired by, and originally based upon, Patternslib and was meant to bring the power of declarative interaction patterns to Plone.

When Mockup started, Patternslib was undergoing significant refactoring and development and it was decided that Mockup should fork and go its own way.

What this means is that the 4 different components mentioned above, were all changed and due to these changes the Mockup project diverged from Patternslib and started developing in a similar but different direction.

So what's the problem?

While Mockup was being developed for the upcoming Plone 5 release, we at Syslab continued using and improving Patternslib in our projects.

Syslab built an intranet for the Star Alliance, which was based on a prototype design by Cornelis Kolbach, the conceptual creator of Patternslib. This design became the inspiration and blueprint for the Plone Intranet Consortium (PIC), which consists of 12 companies working together in a consortium to build an intranet solution on top of Plone.

So, the PIC are building a product using Patternslib, while Plone 5 itself is being built with Mockup, an incompatible fork of Patternslib.

This was clearly a bad situation because we now had:

  • Two incompatible Javascript frameworks being used with Plone.

    Not only were the two frameworks incompatible in the sense that patterns written for the one don't work on the other, but they could also not be used on the same page since the two implementations would compete with one another in invoking Javascript to act upon the same DOM elements.

  • Duplication of effort

    The same or similar patterns were being developed for both frameworks, and when one framework had a pattern which the other wanted, it could only be used after being modified such that it couldn't be used in the original framework anymore.

  • A splitting of the available workforce.

    Developers were either working on Mockup or Patternslib, but almost never on both, which meant that the expertise and experience of developers wasn't being shared between the two projects.

How could this be fixed?

To solve the 3 main problems mentioned above, we needed to merge the common elements of Mockup (specifically the registry, scanner and parser) back into Patternslib.

This will allow developers from both projects to work on the same codebase and enable us to use patterns from both projects together.

At the Alpine City Sprint in Innsbruck, I worked on achieving these goals.

Changes brought in by Mockup

After the fork, Mockup introduced various changes and features which set it apart from Patternslib.

In order to merge Mockup back into Patternslib, I studied these changes and with the help of others came up with strategies on what needed to be done.

Here are some differences and what was done about them:

Mockup allows patterns to also be configured via JSON, whereas Patternslib used a keyword:argument; format

A week before the sprint I added JSON parsing ability to the Patternslib parser, thereby resolving this difference.

Leaves first parsing versus root first parsing

Mockup parses the DOM from the outside in ("root first"), while Patternslib parses the DOM from the inside out ("leaves first").

According to Rok Garbas, the creator of Mockup, the outside-in parsing was done because it reduced complexity in the scanner and the individual patterns.

Wichert Akkerman who originally wrote the Patternslib scanner however provided IMO a very good reason why he chose "leaves first" DOM scanning:

If I remember correctly there are several patterns that rely on any changes in child nodes to have already been made.This is true for several reasons: 1) a pattern may want to attach event handlers to child nodes, which will break of those child nodes are later replaced, and 2) child nodes changing size might impact any measurements made earlier.

Indeed, while refactoring the Mockup code during the merge, I ran into just such a case where a pattern couldn't yet be initialized because patterns inside it weren't yet initialized. By turning around the order of DOM scanning, this problem was resolved and the code in that pattern could be simplified.

So, now after the merge, scanning is "leaves-first" for Mockup patterns as well.

Mockup patterns are extended from a Base object, very similar to how Backbone does it

Patternslib patterns on the other hand are simple Javascript objects without constructors.

The patternslib patterns are conceptually very simple and more explicit.

However, what I like about the Mockup approach is that you have a separate instance with its own private closure for each DOM element for which it is invoked.

After merging, we now effectively have two different methods for writing patterns for Patternslib. The original "vanilla" way, and the Mockup way.

The Mockup parser was completely rewritten

The Mockup parser looks nothing like the Patternslib one and also supports one less configuration syntax (the so-called "shorthand" notation).

This was one piece of code which could not be merged within the time available at the Alpine City Sprint.

So currently we still have two different argument parsers. The Patternslib parser needs to be configured much more expicitly, while the Mockup one is more implicit and makes more assumptions.

Merging these two parsers will probably have to be done at some future sprint.

There are some other more minor differences, such as that every Mockup pattern is automatically registered as a jQuery plugin, but merging these was comparatively easier and I'll won't go into further detail on them.

What I did during the Alpine Sprint

So, in summary:

I refactored the Patternslib registry, scanner and some core utils to let them handle Mockup patterns as well. Luckily the eventual patch for this was quite small and readable.

I changed the Mockup Base pattern so that patterns derived from it now rely on the registry and scanner from Patternslib.

I fixed lots of tests and also wrote some new tests.

This whole task would have been much more difficult and error prone if either Patternslib or Mockup had fewer tests. The Mockup team deserves praise for taking testing very seriously and this allowed me to refactor and merge with much more confidence.

What else is there to still be done?

Being able to use patterns from both projects and merging most of the forked code was a big win, but there are still various things that need to be done to make the merge complete and viable.

I've ranked them from what I think is most important to least important.

1. Update the documentation

Currently documentation is scattered and silo'd in various places (the Patternslib website, the Plone Intranet developer docs and the Mockup developer docs).

The Mockup docs are now out of date ater this merge and need to be brought up to date on these recent changes.

The Patternslib docs are less of a problem because they don't have to deal with Mockup (which can now be seen as an enhancement suite for it), but they can definitely still be improved upon, specifically with an eye on Mockup developers who will start relying on them.

The Plone Intranet consortium also has a useful walkthrough explaining how to create a Patternslib pattern from scratch..

2. Devise a way to also use the Patternslib parser for Mockup patterns

As mentioned, the Mockup patterns still use their own argument parser.

Letting them use the Patternslib's parser will either require extending the Mockup Base pattern to configure the Patternslib parser on a pattern's behalf or instead doing it explicitly in each of the individual patterns.

3. Decide on coding and configuration standards

Unfortunately the coding standards between the two projects differ significantly.

  • The Mockup developers use camelCase as declarative argument names while Patternslib uses dash-separated names.
  • The Mockup source code uses 2 spaces for indentation, Patternslib uses 4 spaces.

4. Remove unnecessary, duplicated patterns

Both projects have patterns for modals, injection, select2 and file upload. These should be merged to get rid of duplication.

5. Move generic patterns (e.g. pat-pickadate, pat-moment etc.) out of Mockup

Mockup has some generic patterns which might also be useful as Patternslib core patterns, in which case they should ideally be moved there.


The sprint was a big success and I'm very happy that all the work has already been merged into the master branches of the matternslib, mockup and mockup-core repositories.

Core code from Mockup is now succesfully merged back into Patternslib and we can use patterns from both projects in the same site.

I had a lot of fun working with talented and motivated software developers and had a lot of opportunities to improve my German.

Thanks again Jens and Christine for organising a great sprint and I look forward to doing such a sprint again!

Innsbruck, the alpine city
Read and Post Comments

Next Page »