Introducing Converse 11: A Major Leap Forward for XMPP Webchat

I’m thrilled to announce the official release of Converse version 11.0.0! 🚀

You can try it out online at chat.conversejs.org or download the desktop app (the Tauri based releases are coming soon).

First and foremost, thanks to all the contributors whose efforts helped make this release possible.

Many people provided code contributions, bug reports, translations, feature requests or just hung out in our chatroom to provide feedback.

I ran the numbers comparing this release to the previous major release.

The previous major release was 933 days ago, that’s about 2.5 years, and over a thousand commits have been made since then. Some of these commits made it into subsequent minor 10.x.x releases, but there’s no denying that the 11.0.0 release is a big one. A quick look at the release notes also shows how much work has gone into it.

Running git rev-list --count v10.0.0..HEAD
1062

I’m particularly excited about this release, because it addresses some long-standing painpoints and issues.

A friend of mine likes to talk about “embarrassment-driven development”.

That’s a bit how I feel about this release. It fixes some problems that have been particularly glaring and embarrassing since a number of years.

Let’s take a closer look at some of the most exciting features and improvements in this latest release:

TypeScript

I’ve always taken a fairly conservative approach with Converse, trying to avoid jumping on any bandwagons and generally trying to evolve the codebase to keep up with the times, instead of opting for huge rewrites.

This project started in 2013, JavaScript developers who have been around since that time, will appreciate the massive amount of churn and changes that have happened in our ecosystem from then until now! There’s even a term coined for it: “JavaScript fatigue”.

I was wary of just rewriting everything in TypeScript. I had limited time to work on Converse, and didn’t want to spend all of it on a rewrite, instead of adding new features and making bugfixes.

Reading about the decision of the team behind Svelte, to use JSDoc comments to add types to their codebase, gave me confidence to try the same with the Converse codebase. I felt that this would be a less disruptive change than trying to rewrite everything in TypeScript, and would fit nicely with the approach I’ve taken until now.

Version 11.0.0 is the first release that includes TypeScript type definitions for Converse.

The tsc compiler is still not running in strict mode, and there are still untyped sections of the code, but it’s a good start, and we’re already reaping benefits.

Virtualized chat history

This is one update I’m particularly proud of. Until now, Converse was rendering all messages it knew about in a particular chat.

If a chat had 2000 messages, all those messages would be rendered, causing all kinds of performance issues.

List virtualization is a technique whereby only a subset of a list is rendered, while the impression is maintained that the entire list is shown.

Converse finally has list virtualization for the chat history.

Mind the gap

Related to history or list virtualization, another update to Converse is to better deal with gaps in the chat history.

Until now, when opening a chat, the entire history from the previous session until now was fetched and shown. When opening Converse in a browser where it didn’t run for, say, 8 months, everything would crawl to a halt as Converse tried to fetch the entire history.

I fixed this, but only fetching the most recent messages, and then placing a special placeholder element in the gap where there are still missing messages.

I then used the IntersectionObserver API, to identify when this gap becomes visible, which would happen example when the user scrolls up to read older messages. As soon as this gap became visible, Converse would fetch a limited number of the most recent messages in that gap, and then place a new placeholder above them, in case there were still missing messages.

In this way, we avoid fetching all messages immediately, but instead we load them lazily as the user scrolls up to read older messages.

I’m happy with the usability and performance improvements this change has brought. One drawback, is that we can’t inform the user of being mentioned, if he or she was mentioned in a message inside that gap of unfetched messages.

This is however to a large extent a limitation of XMPP’s message archiving extension (MAM).

An attempt has been made to address this limitation with the Inbox XEP, but it hasn’t seen any real uptake, particularly among server developers.

Hopefully this drawback will be addressed in due time.

PEP Bookmarks

Adding support for PEP (Personal Eventing Protocol) bookmarks, was some unglamorous modernization work that was also long overdue.

Converse was using an older bookmarks protocol extension, which was causing compatibility issues with other XMPP clients, unless the user’s XMPP server provided a compatibility layer between the two protocols.

By updating to the PEP standard, these issues have now been resolved.

Additionally, Converse now no longer lets the user manually bookmark or unbookmark chats. By doing so, it was unnecessarily exposing protocol details to users, making the app more confusing and difficult to use.

Instead, bookmarks are automatically set and unset as users join and leave groupchats, similarly to how some other XMPP clients already do it.

Updated retractions and moderation

Converse has supported retracting (i.e. deleting) and moderating (i.e. deletion by a moderator) of messages since a number of years already. In the meantime, these protocol extensions have been updated based on community feedback. This release has incorporated those changes.

MUC Private Messages

Finally Converse supports private messages in groupchats.

This is another one of those embarrassment-driven features. Ideally Converse would have had this feature long ago already, but hey, better late then never.

What’s also nice is that some of the work done for this feature has opened up the door for Slack-like threads.

Tagged template literals to render XML stanzas

This is more a new feature I made to Strophe, the underlying XMPP library that Converse uses, but it’s still something pretty cool.

Previously, Strophe’s API for generating XML stanzas, looked like this:

const msg = $msg({
    to: buddy_jid,
    type: 'chat',
    id: u.getUniqueId()
}).c('body').t('Hello world').up()
    .c('active', {'xmlns': 'http://jabber.org/protocol/chatstates'});

Now, with the magic of JavaScript’s tagged template literals, you can write them like this:

const msg = stx`
    <message xmlns="jabber:client" type="chat" id="${u.getUniqueId()}">
        <body>Hello world</body>
        <active xmlns="http://jabber.org/protocol/chatstates"/>
    </message>`;

Much nicer and more readable to my eye. And it’s all standard JavaScript, no compiler necessary.

Bootstrap 5

Converse now uses Bootstrap 5, and the markup has been updated to follow the best practices and conventions for Bootstrap 5.

Right to left support

Converse is translated into over 40 languages. Four of these languages are displayed right to left, namely Arabic, Farsi, Hebrew and Uyghur.

When one of these languages is used (e.g. based on the user’s browser language), Converse will render its UI inverted, from right to left, instead of the usual left to right.

Future directions

There are many features and improvements that could still be developed for a future release.

Some highlights include:

• Microblogging (Github issue, XEP-0277 and XEP-0472).
• Slack-like message threads.
• Push notifications (Github issue)
• A mobile app (Github issue)
• Audio/video calls integration (Github issue)

If you’re considering sponsoring any of these features, please reach out to me via the contact form.

I’m also currently looking for shorter term freelance contracts, whether Converse-related or not. If you’re interested in hiring me, please reach out.

We also accept general donations and sponsorships via Patreon, Liberapay and Github. Sponsors who donate $100 or more per month will have their logo displayed on the Converse website and app.

Fin

That’s all, thanks for reading and I hope you enjoy the new release.

Hello, I'm JC Brand, software developer and consultant.
I have decades of experience working with open source software, for governments, small startups and large corporates.
I created and maintain Converse, a popular XMPP chat client.
I can help you integrate chat and instant messaging features into your website or intranet.

Don't hesitate to contact me if you'd like to connect.