blog

Report on the Dusseldorf XMPP sprint

| categories: xmpp, sprint, dusseldorf

Recently we held another XMPP sprint, this time in Dusseldorf at the Chaosdorf clubhouse.

We had 10 participants and an ambitious agenda with a long list of proposed topics to sprint on. We didn't get to work on everything, and we spent a considerable amount of time discussing and prioritizing the agenda. Some of the things not worked on can however easily be moved to future sprints.

After the sprint, iNPUTmice suggested that we consider having more focused sprints, for example focusing only on implementing XEP-369 MIX in clients and servers.

I think having a more focused, single-topic sprint is a great idea, and MIX is a good topic to tackle. AFAIK there's currently no FOSS MIX implementation for either clients or servers. IMO this can partly be attributed to a stalemate situation where everyone is waiting for someone else to make the first move. By organizing a sprint around it we can break the deadlock by starting to work on it together.

Discussions

XEP-0283 <moved/> (pep., Ge0rG)

XEP-0283 provides a mechanism for informing your contacts that you're changed your account and now have a new JID.

pep. and Ge0rG had a discussion on how to automate the operation entirely.

Currently the security considerations contain a clause which to them defeats the point of the XEP:

"In order to prevent other users from maliciously altering contacts the client SHOULD NOT automatically subscribe to a <moved/> JID when it receives an unsubscribe and SHOULD NOT automatically unsubscribe to a <moved/> JID when it receives a subscribe."

Also the XEP uses presence stanza only, which makes the whole operation rather ephemeral.

The plan is to change the XEP to use PEP mostly.

Programming, documentation and various other tasks

iNPUTmice added two new tests to the compliance tester, one for MUC Avatars and a check for CORS Headers on HTTP Upload, which is especially important for webchat clients like Converse.

He also tried to fix a bug in Conversations that happens intermittently on Android 9.

MattJ worked on the 0.11 release of Prosody. He also worked on adding support for CORS in mod_http_upload.

pep. registered an XMPP assembly for 35C3. He also worked on the slixmpp OMEMO plugin and got decryption working. Additionally he and Link Mauve worked on a French translation of the Prosody release announcement.

Link Mauve worked on implementing Jingle in Converse, fixing bugs and updating strophe.jingle.js to not use jQuery. As part of this, he and I looked at better exception handling in Converse.

He also helped pep. and syndace with the slixmpp plugin interface and spent some time trying to deploy JSXC.

Roel worked on the homebrewserver.club article on how to configure a modern XMPP server. MattJ helped out on Sunday as well and AFAIK their collaboration also caused MattJ to add some clarifications and simplifications to the Prosody config file.

Roel and rtq3 also worked on a Spanish translation of the homebrewserver.club article.

Syndace worked on resurrecting a non-merged PR from Andy Straub for the OMEMO XEP which adds various changes to the XEP, including using an XML-based wire format. By choosing our own wire format, we also make permissive OSS licensed OMEMO implementations possible. Such a change will however not be backwards compatible with current OMEMO implementations.

Andy joined us on Sunday and participated in discussions around the PR and OMEMO in general with iNPUTmice, pep and Syndace.

Haeckse showed us a demo of Chatty, the chat (and SMS) client that she is working on for the upcoming Purism phone.

I worked on adding support for XEP-0156 to Converse, based on a PR from Link Mauve. The original PR was closed, due to being very old and difficult to merge, and work is now continuing in a branch with a new PR #1340.

I also made smaller fixes and changes, for example only rendering image URLs inline if they use HTTPS (to avoid mixed-content warnings in the browser).

Conclusion

Chaosdorf

Much thanks to Chaosdorf, for generously making their clubhouse available for the sprint.

Kuro even made some chilli for lunch and baked a cake!

In the evenings we went out to dinner. Dusseldorf has a large Japanese population, so we were quite keen to try out some Japanese food.

I had proper Ramen for the first time on Sunday night and on Saturday we had some great sushi.

Future sprints

A sprint can be measured by its output in the form of code, documentation, translations and so forth, but this only tells part of the story.

Much of what makes sprints valuable is the social aspect of making friends and working together on something you all care about.

One way to judge the success of the sprint, is to consider how excited the participants are about organising and attending future sprints.

Roel intends to organize a sprint in Brussels, around the time of the XMPP summit and FOSDEM. This sprint will focus on UI/UX issues and on finding a common user-facing terminology for XMPP clients. Roel comes from a design background and will be inviting some fellow like-minded designers to join.

Link Mauve indicated that he'd like to organize a sprint in Paris in 2019 and pep. thinks we should go to Sweden.

During our sprint we also found out that the Berlin XMPP Meetup group is considering organizing a sprint there soon.

Then there's also the upcoming 35C3, where there will be an XMPP assembly and the opportunity to hack together.

Thanks to everyone who joined the Dusseldorf sprint, some of whom spent a considerable sum of money in order to be there. I had a great time and am looking forward to more such events coming in 2019.

Come chat with us

We hang out in the XMPP-Sprint groupchat and this is also where upcoming XMPP sprints will be discussed and planned.

You can visit the webchat at https://chat.cluxia.eu/anon/#xmpp-sprint or click on xmpp:xmpp-sprint@chat.cluxia.eu?join to open it in an installed client.

---

Dusseldorf XMPP Sprint, 16 and 17 January 2018

Sprint Report: Merging Mockup and Patternslib

| categories: mockup, javascript, patternslib, austria, sprint, foss, plone

Alpine City Sprint, 21 to 26 January 2015

This is a report on what I did at the Plone Alpine City Sprint organized by Jens Klein and Christina Baumgartner in Innsbruck between 21 and 26 January 2015.

Firstly, I want to thank to them for organizing and hosting a great sprint and for being such friendly and generous hosts.

My goal for this sprint was to work on merging the Patternslib and Mockup Javascript frameworks and I'm happy to report that very good progress was made.

Long story short, the merge was successful and it's now possible to use patterns written for either project within a single framework.

Before I expand on how this was achieved during this sprint, I'll first provide some necessary background information to place things into context and to explain why this work was necessary.

What is Patternslib?

Patternslib brings webdesign and development together.

Patternslib's goal is to allow website designers to build rich interactive prototypes without having to write any Javascript. Instead the designer simply writes normal HTML and then adds dynamism and interactivity by adding special HTML classes and data-attributes to selected elements.

For example, here is a Pattern which injects a list of recent blog posts into a element in this page:

Click here to show recent blog posts

The declarative markup for this pattern looks as follows:

<section id="alpine-blog-injected">
    <a href="#portlet-recent-blogs"
       class="pat-inject"
       data-pat-inject="target: #alpine-blog-injected">

        Click here to show recent blog posts
    </a>
</section>

The dynamic behavior comes from the fact that I have the Patternslib Javascript library loaded in this page.

On pageload, Patternslib scans the DOM looking for elements which declare any of the already registered patterns. If it finds such an element, it invokes a Javacript module associated with that pattern.

In the above example, the <a> element declares that it wants the Inject pattern applied to it, by virtue of it having the pat-inject HTML class.

Patterns are configured with specific HTML5 data properties. In the example above, it is the data-pat-inject property which specifies that the target for injection is the #alpine-blog-injected element in the current page and the content being injected is specified by the href attribute of the <a> element. In this case, the href points to an anchor inside the current page, but it might just as well be a link to another page which will then get injected via Ajax.

More information on how to configure pat-inject can be found on the patternslib website.

Each pattern has a corresponding Javascript module which is often just a wrapper around an existing Javascript library (such as Parsley, Select2 or Pickadate) that properly defines the configuration options for the pattern and properly invokes or applies the 3rd party library.

So, in order to do all this, Patternslib can be broken down into the folowing components:

  • A registry which lists all the available patterns.
  • A scanner which scans the DOM to identify declared patterns.
  • A parser which parses matched DOM elements for configuration settings.
  • The individual Javascript modules which implement the different patterns.

What is Mockup?

Mockup, declarative interaction patterns for Plone..

Mockup was inspired by, and originally based upon, Patternslib and was meant to bring the power of declarative interaction patterns to Plone.

When Mockup started, Patternslib was undergoing significant refactoring and development and it was decided that Mockup should fork and go its own way.

What this means is that the 4 different components mentioned above, were all changed and due to these changes the Mockup project diverged from Patternslib and started developing in a similar but different direction.

So what's the problem?

While Mockup was being developed for the upcoming Plone 5 release, we at Syslab continued using and improving Patternslib in our projects.

Syslab built an intranet for the Star Alliance, which was based on a prototype design by Cornelis Kolbach, the conceptual creator of Patternslib. This design became the inspiration and blueprint for the Plone Intranet Consortium (PIC), which consists of 12 companies working together in a consortium to build an intranet solution on top of Plone.

So, the PIC are building a product using Patternslib, while Plone 5 itself is being built with Mockup, an incompatible fork of Patternslib.

This was clearly a bad situation because we now had:

  • Two incompatible Javascript frameworks being used with Plone.

    Not only were the two frameworks incompatible in the sense that patterns written for the one don't work on the other, but they could also not be used on the same page since the two implementations would compete with one another in invoking Javascript to act upon the same DOM elements.

  • Duplication of effort

    The same or similar patterns were being developed for both frameworks, and when one framework had a pattern which the other wanted, it could only be used after being modified such that it couldn't be used in the original framework anymore.

  • A splitting of the available workforce.

    Developers were either working on Mockup or Patternslib, but almost never on both, which meant that the expertise and experience of developers wasn't being shared between the two projects.

How could this be fixed?

To solve the 3 main problems mentioned above, we needed to merge the common elements of Mockup (specifically the registry, scanner and parser) back into Patternslib.

This will allow developers from both projects to work on the same codebase and enable us to use patterns from both projects together.

At the Alpine City Sprint in Innsbruck, I worked on achieving these goals.

Changes brought in by Mockup

After the fork, Mockup introduced various changes and features which set it apart from Patternslib.

In order to merge Mockup back into Patternslib, I studied these changes and with the help of others came up with strategies on what needed to be done.

Here are some differences and what was done about them:

Mockup allows patterns to also be configured via JSON, whereas Patternslib used a keyword:argument; format

A week before the sprint I added JSON parsing ability to the Patternslib parser, thereby resolving this difference.

Leaves first parsing versus root first parsing

Mockup parses the DOM from the outside in ("root first"), while Patternslib parses the DOM from the inside out ("leaves first").

According to Rok Garbas, the creator of Mockup, the outside-in parsing was done because it reduced complexity in the scanner and the individual patterns.

Wichert Akkerman who originally wrote the Patternslib scanner however provided IMO a very good reason why he chose "leaves first" DOM scanning:

If I remember correctly there are several patterns that rely on any changes in child nodes to have already been made.This is true for several reasons: 1) a pattern may want to attach event handlers to child nodes, which will break of those child nodes are later replaced, and 2) child nodes changing size might impact any measurements made earlier.

Indeed, while refactoring the Mockup code during the merge, I ran into just such a case where a pattern couldn't yet be initialized because patterns inside it weren't yet initialized. By turning around the order of DOM scanning, this problem was resolved and the code in that pattern could be simplified.

So, now after the merge, scanning is "leaves-first" for Mockup patterns as well.

Mockup patterns are extended from a Base object, very similar to how Backbone does it

Patternslib patterns on the other hand are simple Javascript objects without constructors.

The patternslib patterns are conceptually very simple and more explicit.

However, what I like about the Mockup approach is that you have a separate instance with its own private closure for each DOM element for which it is invoked.

After merging, we now effectively have two different methods for writing patterns for Patternslib. The original "vanilla" way, and the Mockup way.

The Mockup parser was completely rewritten

The Mockup parser looks nothing like the Patternslib one and also supports one less configuration syntax (the so-called "shorthand" notation).

This was one piece of code which could not be merged within the time available at the Alpine City Sprint.

So currently we still have two different argument parsers. The Patternslib parser needs to be configured much more expicitly, while the Mockup one is more implicit and makes more assumptions.

Merging these two parsers will probably have to be done at some future sprint.

There are some other more minor differences, such as that every Mockup pattern is automatically registered as a jQuery plugin, but merging these was comparatively easier and I'll won't go into further detail on them.

What I did during the Alpine Sprint

So, in summary:

I refactored the Patternslib registry, scanner and some core utils to let them handle Mockup patterns as well. Luckily the eventual patch for this was quite small and readable.

I changed the Mockup Base pattern so that patterns derived from it now rely on the registry and scanner from Patternslib.

I fixed lots of tests and also wrote some new tests.

This whole task would have been much more difficult and error prone if either Patternslib or Mockup had fewer tests. The Mockup team deserves praise for taking testing very seriously and this allowed me to refactor and merge with much more confidence.

What else is there to still be done?

Being able to use patterns from both projects and merging most of the forked code was a big win, but there are still various things that need to be done to make the merge complete and viable.

I've ranked them from what I think is most important to least important.

1. Update the documentation

Currently documentation is scattered and silo'd in various places (the Patternslib website, the Plone Intranet developer docs and the Mockup developer docs).

The Mockup docs are now out of date ater this merge and need to be brought up to date on these recent changes.

The Patternslib docs are less of a problem because they don't have to deal with Mockup (which can now be seen as an enhancement suite for it), but they can definitely still be improved upon, specifically with an eye on Mockup developers who will start relying on them.

The Plone Intranet consortium also has a useful walkthrough explaining how to create a Patternslib pattern from scratch..

2. Devise a way to also use the Patternslib parser for Mockup patterns

As mentioned, the Mockup patterns still use their own argument parser.

Letting them use the Patternslib's parser will either require extending the Mockup Base pattern to configure the Patternslib parser on a pattern's behalf or instead doing it explicitly in each of the individual patterns.

3. Decide on coding and configuration standards

Unfortunately the coding standards between the two projects differ significantly.

  • The Mockup developers use camelCase as declarative argument names while Patternslib uses dash-separated names.
  • The Mockup source code uses 2 spaces for indentation, Patternslib uses 4 spaces.

4. Remove unnecessary, duplicated patterns

Both projects have patterns for modals, injection, select2 and file upload. These should be merged to get rid of duplication.

5. Move generic patterns (e.g. pat-pickadate, pat-moment etc.) out of Mockup

Mockup has some generic patterns which might also be useful as Patternslib core patterns, in which case they should ideally be moved there.

Conclusion

The sprint was a big success and I'm very happy that all the work has already been merged into the master branches of the matternslib, mockup and mockup-core repositories.

Core code from Mockup is now succesfully merged back into Patternslib and we can use patterns from both projects in the same site.

I had a lot of fun working with talented and motivated software developers and had a lot of opportunities to improve my German.

Thanks again Jens and Christine for organising a great sprint and I look forward to doing such a sprint again!

Innsbruck, the alpine city

Report on the Beer and Wine Sprint in Cape Town

| categories: south-africa, plone, sprint

The Beer and Wine sprint wrapped up on Sunday and it's now a good time to report on what was South Africa's first ever core Plone sprint. This was a joint sprint between Cape Town and Munich, which in hindsight, was a very good idea. The two groups of sprinters had daily meetings via Google hangouts, and the structure and support we received from Munich helped a lot. Connecting with the sprinters on another continent brought home the point that we are a global community and that no core Plone work is done in isolation. The daily hangouts also motivated us to get cracking so that we could have something to report on!

/img/209_4385.JPG

South Africa is geographically isolated from the places where most Plone development takes place (notably Europe and North America) , which results in the local Plone community not getting as much exposure to sprints and community events as would have been possible otherwise. The goal of this sprint was to help change that, and to provide an opportunity for us to contribute back to the Plone community from which we've received so much.

Most of the people who attended the sprint were first-timers, and it was a joy to see the enthusiasm with which people started working. Eric Steele and David Glick flew down to join us here at the tip of Africa, and provided much needed experience and support.

/img/209_4389.JPG

About 14 people met at sprint venue on Thursday morning and after introductions and a quick hello to the Europeans via the wormhole, we formed groups and started sprinting.

/img/209_4380.JPG

Mike Metcalfe and Campbell Mckellar-Bassset (rockfruit) started working on porting CMFPlone tests from Products.PloneTestCase to plone.app.testing. They kept at it with dogged determination for the next four days of the sprint and made good progress. Izak Burger and Jurgen Blignaut tackled some TinyMCE bugs, while Rudi Ackhurst, Witek, Johan Beyers and myself started working on Plip #13260. We did this via Dojo sessions, which we held for most of the sprint. Other people did tutorials and proofreading of documentation.

It appears as if the sprinting bug has bitten at least some of the people who attended, and there's already been talk of doing this again some time. I personally had an amazing experience and learned a lot while organising this event.

/img/8415489149_4d6343914c.jpg

To the bittereinders who sprinted every single day (Thursday to Sunday), thanks for your energy and support! This would have not been possible without you. We laid a solid foundation for future core Plone contributions from our local group.

Lastly, thanks a million to everyone involved in making this sprint a reality, Eric, David and all the people who attended, our sponsors the Praekelt and Plone Foundations, the guys in Munich and all those who helped with support and advice. And thanks to Liz Leddy who planted the initial seed in my head that this was not only possible, but a actually great idea!

P.S Credit for 3 of the photos goes to David Glick

/img/8416589378_ffb70c74dc_c.jpg