Sprint Report: Merging Mockup and Patternslib

This is a report on what I did at the Plone.

Alpine City Sprint organized by Jens Klein and Christina Baumgartner in Innsbruck between 21 and 26 January 2015.

Firstly, I want to thank to them for organizing and hosting a great sprint and for being such friendly and generous hosts. My goal for this sprint was to work on merging the Patternslib and Mockup Javascript frameworks and I’m happy to report that very good progress was made.

Long story short, the merge was successful and it’s now possible to use patterns written for either project within a single framework. Before I expand on how this was achieved during this sprint, I’ll first provide some necessary background information to place things into context and to explain why this work was necessary.

What is Patternslib?

Patternslib’s goal is to allow website designers to build rich interactive prototypes without having to write any Javascript. Instead the designer simply writes normal HTML and then adds dynamism and interactivity by adding special HTML classes and data-attributes to selected elements.

For example, here is a Pattern which injects a list of recent blog posts into a element in this page:

The declarative markup for this pattern looks as follows:

<section id="alpine-blog-injected">
    <a href="#portlet-recent-blogs"
       class="pat-inject" data-pat-inject="target: #alpine-blog-injected">
        Click here to show recent blog posts
    </a>
</section>

The dynamic behavior comes from the fact that I have the Patternslib Javascript library loaded in this page.

On pageload, Patternslib scans the DOM looking for elements which declare any of the already registered patterns. If it finds such an element, it invokes a Javacript module associated with that pattern.

In the above example, the <a> element declares that it wants the Inject pattern applied to it, by virtue of it having the pat-inject HTML class.

Patterns are configured with specific HTML5 data properties. In the example above, it is the data-pat-inject property which specifies that the target for injection is the #alpine-blog-injected element in the current page and the content being injected is specified by the href attribute of the <a> element. In this case, the href points to an anchor inside the current page, but it might just as well be a link to another page which will then get injected via Ajax.

More information on how to configure pat-inject can be found on the patternslib website.

Each pattern has a corresponding Javascript module which is often just a wrapper around an existing Javascript library (such as Parsley, Select2 or Pickadate) that properly defines the configuration options for the pattern and properly invokes or applies the 3rd party library.

So, in order to do all this, Patternslib can be broken down into the folowing components:

• A registry which lists all the available patterns.
• A scanner which scans the DOM to identify declared patterns.
• A parser which parses matched DOM elements for configuration settings.
• The individual Javascript modules which implement the different patterns.

What is Mockup?

Mockup was inspired by, and originally based upon, Patternslib and was meant to bring the power of declarative interaction patterns to Plone.

When Mockup started, Patternslib was undergoing significant refactoring and development and it was decided that Mockup should fork and go its own way.

What this means is that the 4 different components mentioned above, were all changed and due to these changes the Mockup project diverged from Patternslib and started developing in a similar but different direction.

So what’s the problem?

While Mockup was being developed for the upcoming Plone 5 release, we at Syslab continued using and improving Patternslib in our projects.

Syslab built an intranet for the Star Alliance, which was based on a prototype design by Cornelis Kolbach, the conceptual creator of Patternslib. This design became the inspiration and blueprint for the Plone Intranet Consortium (PIC), which consists of 12 companies working together in a consortium to build an intranet solution on top of Plone.

So, the PIC are building a product using Patternslib, while Plone 5 itself is being built with Mockup, an incompatible fork of Patternslib.

This was clearly a bad situation because we now had:

• Two incompatible Javascript frameworks being used with Plone.

  Not only were the two frameworks incompatible in the sense that patterns written for the one don’t work on the other, but they could also not be used on the same page since the two implementations would compete with one another in invoking Javascript to act upon the same DOM elements.

• Duplication of effort

  The same or similar patterns were being developed for both frameworks, and when one framework had a pattern which the other wanted, it could only be used after being modified such that it couldn’t be used in the original framework anymore.

• A splitting of the available workforce.

  Developers were either working on Mockup or Patternslib, but almost never on both, which meant that the expertise and experience of developers wasn’t being shared between the two projects.

How could this be fixed?

To solve the 3 main problems mentioned above, we needed to merge the common elements of Mockup (specifically the registry, scanner and parser) back into Patternslib.

This will allow developers from both projects to work on the same codebase and enable us to use patterns from both projects together.

At the Alpine City Sprint in Innsbruck, I worked on achieving these goals.

Changes brought in by Mockup

After the fork, Mockup introduced various changes and features which set it apart from Patternslib.

In order to merge Mockup back into Patternslib, I studied these changes and with the help of others came up with strategies on what needed to be done.

Here are some differences and what was done about them:

• Mockup allows patterns to also be configured via JSON, whereas Patternslib used a keyword:argument; format

  A week before the sprint I added JSON parsing ability to the Patternslib parser, thereby resolving this difference.

• “Leaves first” parsing versus “root first” parsing

  Mockup parses the DOM from the outside in (“root first”), while Patternslib parses the DOM from the inside out (“leaves first”).

  According to Rok Garbas, the creator of Mockup, the outside-in parsing was done because it reduced complexity in the scanner and the individual patterns.

  Wichert Akkerman who originally wrote the Patternslib scanner however provided a good reason why he chose “leaves first” DOM scanning:

  If I remember correctly there are several patterns that rely on any changes in child nodes to have already been made.This is true for several reasons: 1) a pattern may want to attach event handlers to child nodes, which will break of those child nodes are later replaced, and 2) child nodes changing size might impact any measurements made earlier.

  Indeed, while refactoring the Mockup code during the merge, I ran into just such a case where a pattern couldn’t yet be initialized because patterns inside it weren’t yet initialized. By turning around the order of DOM scanning, this problem was resolved and the code in that pattern could be simplified.

  So, now after the merge, scanning is “leaves-first” for Mockup patterns as well.

• Mockup patterns are classes extended from a Base object

  Patternslib patterns on the other hand are simple JavaScript objects without constructors.

  The Patternslib patterns are conceptually very simple and more explicit.

  However, what I like about the Mockup approach is that you have a separate instance with its own private closure for each DOM element for which it is invoked.

  After merging, we now effectively have two different methods for writing patterns for Patternslib. The original “vanilla” way, and the Mockup way.

• The Mockup parser was completely rewritten

  The Mockup parser looks nothing like the Patternslib one and also supports one less configuration syntax (the so-called “shorthand” notation).

  This was one piece of code which could not be merged within the time available at the Alpine City Sprint.

  So currently we still have two different argument parsers. The Patternslib parser needs to be configured much more expicitly, while the Mockup one is more implicit and makes more assumptions.

  Merging these two parsers will probably have to be done at some future sprint.

There are some other more minor differences, such as that every Mockup pattern is automatically registered as a jQuery plugin, but merging these was comparatively easier and I’ll won’t go into further detail on them.

What I did during the Alpine Sprint

So, in summary:

• I refactored the Patternslib registry, scanner and some core utils to let them handle Mockup patterns as well. Luckily the eventual patch for this was quite small and readable.

• I changed the Mockup Base pattern so that patterns derived from it now rely on the registry and scanner from Patternslib.

• I fixed lots of tests and also wrote some new tests.
  
  This whole task would have been much more difficult and error prone if either Patternslib or Mockup had fewer tests. The Mockup team deserves praise for taking testing very seriously and this allowed me to refactor and merge with much more confidence.

What else is there to still be done?

Being able to use patterns from both projects and merging most of the forked code was a big win, but there are still various things that need to be done to make the merge complete and viable.

I’ve ranked them from what I think is most important to least important.

1. Update the documentation

Currently documentation is scattered and silo’d in various places (the Patternslib website, the Plone Intranet developer docs and the Mockup developer docs).

The Mockup docs are now out of date ater this merge and need to be brought up to date on these recent changes.

The Patternslib docs are less of a problem because they don’t have to deal with Mockup (which can now be seen as an enhancement suite for it), but they can definitely still be improved upon, specifically with an eye on Mockup developers who will start relying on them.

The Plone Intranet consortium also has a useful walkthrough explaining how to create a Patternslib pattern from scratch..

2. Devise a way to also use the Patternslib parser for Mockup patterns

As mentioned, the Mockup patterns still use their own argument parser.

Letting them use the Patternslib’s parser will either require extending the Mockup Base pattern to configure the Patternslib parser on a pattern’s behalf or instead doing it explicitly in each of the individual patterns.

3. Decide on coding and configuration standards

Unfortunately the coding standards between the two projects differ significantly.

• The Mockup developers use camelCase as declarative argument names while Patternslib uses dash-separated names.
• The Mockup source code uses 2 spaces for indentation, Patternslib uses 4 spaces.

4. Remove unnecessary, duplicated patterns

Both projects have patterns for modals, injection, select2 and file upload. These should be merged to get rid of duplication.

5. Move generic patterns (e.g. pat-pickadate, pat-moment etc.) out of Mockup

Mockup has some generic patterns which might also be useful as Patternslib core patterns, in which case they should ideally be moved there.

Conclusion

The sprint was a big success and I’m very happy that all the work has already been merged into the master branches of the matternslib, mockup and mockup-core repositories.

Core code from Mockup is now succesfully merged back into Patternslib and we can use patterns from both projects in the same site.

I had a lot of fun working with talented and motivated software developers and had a lot of opportunities to improve my German.

Thanks again Jens and Christine for organising a great sprint and I look forward to doing such a sprint again!

Hello, I'm JC Brand, software developer and consultant.
I created and maintain Converse, a popular web-based XMPP chat client,
I can help you integrate chat and instant messaging features into your website or intranet.

You can follow me on the Fediverse or on Twitter.