blog

Open Source software and the expection of free labor

| categories: foss, open-source, economics | View Comments

Over the last few years of starting and then maintaining an open source project that has received a decent amount of attention, converse.js, I've noticed some interesting things about the expectations some people have towards developers who work on FOSS (free and open source software).

Predictably irrational

Book cover: Predictably irrational

People of course love to receive something for nothing. Dan Ariely, in his book "Predictably Irrational" illustrates some of the biases people have when it comes to free stuff. When confronted with the words "free" (as in gratis), people do things that are irrational and are at odds with how a rational actor (the mythical homo economicus) is expected to behave, which is the bedrock upon which most economic theories are based.

The outcome of the various studies Ariely conducted was consistent: when faced with multiple choices, the free option was commonly chosen. With the opportunity to receive something for free, the actual value of the product or service is no longer considered. [1]

“Most transactions have an upside and a downside, but when something is FREE! we forget the downside. FREE! gives us such an emotional charge that we perceive what is being offered as immensely more valuable than it really is.”

Dan Ariely

The biases regarding "FREE!" apply not only to monetary costs, but also to time. We forgo some of our time when we wait in line for free popcorn or to enter a museum on a free-entrance day. We could have been doing something else at that time, so there's a resultant opportunity cost. [1]

Freedom isn't free, it costs folks like you and me

[2]

These biases of course can also come into play when people evaluate free (as in beer) software. In the same way that people didn't take into consideration the cost of the time they spend in trying to get something for "free", people often also don't consider the non-monetary costs of using FOSS.

A common retort that usually surfaces on Slashdot, Reddit or Hacker News whenever a discussion around using a Linux distribution on the Desktop takes place, is “Linux is only free if you don't value your time”.

That's of course completely true. I do value my time, took that into consideration and still concluded that I want to use GNU/Linux and free and open source software.

Using FOSS requires a certain amount of commitment, and it should be clear to the user why they are willing to go that route (freedom from vendor-locking, the ability to control and keep private your data, the ability to modify the code to your liking etc.).

I think people have been hyping the "FREE!" aspect of FOSS way too much.

Software for nothing and your support for free

[3]

Note

Disclaimer

I consider a certain amount of support and maintenance as a requirement for a successful open source project and not something you (as the developer) can ignore.

I try to channel bug reports and feature requests to the Github issue tracker and general support questions to a mailing list, where hopefully other people would also be willing to share the load by answering questions.

So while I complain about people wanting "something for nothing" below, I invariably mean people who write to me directly, instead of on the issue tracker and who are often trying to get me to work on something right away.

So, when considering that many people don't properly evaluate the costs involved in using FOSS, some requests and emails that I sometimes receive start to make sense.

“Please guide me”

One common recurrence, is to be contacted by someone who is integrating converse.js into a project for a paying client, and somehow got stuck. Perhaps they didn't read the docs or perhaps they don't have the requisite technical skills to do the job. These emails sometimes have a pleading, desparate tone to them. Perhaps to instill some sense of guilt or obligation or perhaps just because the person is really desperate and under time pressure.

What gets me every time however, is that as far as I can tell, these are people working for commercial businesses who get paid for the work they do. They then trawl the web looking for hapless FOSS developers to do their work for them for free, or as expressed in the commonly used phrase in these kinds of emails: “Please guide me”.

The novelty and warm fuzzy feeling of altruistically helping strangers solve their problems disappears like mist before the sun when you realise that they're getting paid for the work you're doing for them right now.

And make no mistake about it, maintenance and support for an open source project is work and sometimes even drudgery. The fun part is writing new code or trying out new things, not helping people who can't be bothered to study the documentation.

We need a feature and we hope you'll do it for free

Another common theme is emails where people somehow just assume that I'll implement some feature for them. At first this presumptuousness startled me.

I think it's totally fair to ask when the project is charitable and the people involved don't receive any payment themselves, but that's often not the case.

Instead, the underlying assumption appears to be that I love working on open source projects so much that I'll do it all for free and that I don't have ideas on what to work on next.

Sometimes people qualify their requests by stating that they're a small non-profit. Non-profits do however pay out salaries, don't they?

I'd be willing to reduce my hourly rate when working for a non-profit with a good cause, but I'm most likely not going to do work for free.

The software is free, but the time spent working on it costs money

A nuance that's perhaps lost on many people, is that I have often worked on converse.js for money. There was a rather long "bootstrapping" phase in the beginning where the project wasn't good enough for anyone to actually use or pay money for further development, but after the project stabilized I started getting small paying gigs of custom development on converse.js.

In all cases I made it clear that the Mozilla Public License forces me to open source any changes I made to the covered files, and therefore the work I did for these paying customers (bless their hearts) was open sourced as well.

The point is that while the software is free (as in beer and as in speech), the time spent working on it costs money.

Either someone else pays me to spend my time working on it, or I end up paying by doing something for free while I might be getting paid doing something else (opportunity costs) or by taking time away from other activities.

FOSS development costs money, either the developer is commissioned, or they pay for it themselves (perhaps unwittingly).

Doing work for free devalues it and takes the piss out of actual paying customers

The last point I'd like to make, is that by taking on these requests to do free work for commercial entities (and non-profits), I'm not only devaluing my work, but I'm also disincentivising paying customers (which includes non-profits).

After all, why would anyone pay me to do anything if I'm so eager to please that I'll do it all for free?

The only reason I could see to do that, is to get that mythical "exposure" that's often also sold to web and graphic designers.

The Oatmeal comic: "Exposure"

[4]

So what do you do if you need work done and can't pay for it?

Free and open source software is a beautiful, world-changing and paradigm shifting idea. However, software developers, like all people, need to be paid for their work, also when they work on FOSS.

If you can't pay for software development, then you can still try to incentivise FOSS developers in other ways, but be aware that it'll be more difficult.

One important lesson that I'm glad I learned early in life, is that when you're asking someone to do something for you, then you need to explain to them why it's in their best interests to do so.

People inherently look out for themselves. It's perfectly natural and doesn't necessarily mean they're selfish to the point of being anti-social, it just means that they need to take care of themselves and that they can't expect other people to do it for them or to even have their best interests at heart.

So when desiring something from someone, such as their help, the best approach is to explain to them what's in it for them

For example, if you want a friend to help you out with something, let's say to join a beach cleanup project, you don't tell them why it'll be good for you, you explain to them that it'll be an opportunity to chat, to meet new people, to go for a swim and to have the enjoyment of a clean unspoiled beach.

This is simple stuff, but many people apparently don't know this.

So if you want someone to help you with a software project, explain to them why it would be in their best interest. If you can't find a reason why, then perhaps it's actually not in their best interest and you need to create an incentive for them.

Money works pretty well as an incentive, but there are other ways as well. One sure-fire way to build up goodwill and gratitude (that might translate into more help and assistance) is to contribute. If you can't write code, fix typos in the docs, evangelize the project or contribute in other areas which you have some expertise, like translations, design, UX, helpful feedback etc.

FOSS development is a community effort and a team sport. There'll always be people who try to take more than they give, but on average, humans are matchers. When given something, they want to reciprocate and give back. Keep that in mind when you're trying to get something for nothing.

References

[1](1, 2) Wikipedia article on Predictable Irrational
[2]Sung to the tune of Freedom isn't free
[3]Sung to the tune of "Money for nothing" by Dire Straits.
[4]From The Oatmeal
Read and Post Comments

The Solution to Jaron Lanier's "Siren Servers"

| categories: economics, foss, open-source, surveillance | View Comments

In his book Jaron Lanier claims that the information on the web is undervalued and underpriced.

According to him this is hollowing out the middle class because people aren't being sufficiently valued for their information-producing work, while a new rich elite is making vast sums of money by collecting and analysing this cheap information.

Who owns the future? by Jaron Lanier

In many cases the information being collected and analysed is given freely to companies by the users themselves in return for email, calendaring and social networking features.

This results in information assymmetry (e.g. who knows what about whom) and therefore a large disparity in power and wealth. Google knows everything about you and yet you know nothing of value about Google.

Siren Servers

These online services are hosted on servers, which Lanier has termed Siren Servers [1], after the Sirens of Greek Mythology. The Sirens are beautiful but dangerous creatures who lure nearby sailors with their enchanting music and voices, thereby letting them shipwreck on the rocky coast of their island.

These Siren Servers lure us into opening up our lives, so that they might freely gather data from us, on a truly massive scale, which they then analyse and profit from.

The results of this analysis is "kept secret and used to manipulate the rest of the world to advantage."

So much value is extracted from this data, that these Siren Servers are valued in the billions of dollars.

The Siren, Edward Armitage, 1888

The Siren, Edward Armitage, 1888

Lanier claims that the users of these servers are shortchanged, in that they don't get their data's worth in return. Not to mention the broader societal damage wrought by the surveillance-as-business-model engendered by this approach as well as the decimation of the middle class as software takes over many industries.

Siren Servers reduce risk for themselves, but in the process increase the risk for all the other smaller players. Uber's business model for example is all about shifting risks onto its drivers. They would not be profitable if they had to carry the costs of commercial insurance, licensing, vehicle maintenance and oversight of drivers.

"The total amount of risk in the market as a whole stays the same, perhaps, but it's not distributed evenly. Instead the smaller players take on more risk while the player with the biggest computer takes on less."

Lanier's solution to the emergence of Siren Servers is to "properly account" for the wealth created by people online. In other words, people need to be paid for what they do online, no matter how small and seemingly insignificant.

"If information age accounting were complete and honest, as much information as possible would be valued in economic terms. If, however "raw" information, or information that hasn't yet been routed by those who run the most central computers, isn't valued, then a massive disenfranchisement will take place."

The decimation of the middle class

Lanier says this process started in the creative industry, with musicians and photographers impacted quite early on, and that it's now spreading to other occupations, such as travel agents, estate agents and journalists.

Steve Albini playing guitar

Steve Albini playing guitar (2007)

Lanier writes:

"Making information free is survivable so long as only limited numbers of people are disenfranchised. As much as it pains me to say so, we can survive if we only destroy the middle classes of musicians, journalists and photographers. What is not survivable is the additional destruction of the middle classes in transportation, manufacturing, energy, office work, education and health care. And all that destruction will come surely if the dominant idea of an information economy isn't improved."

In other words, until we reconfigure the economic system to value raw data by compensating the people that generate it, we will have massive inequality.

Not everybody is of course as pessimistic or draws these same conclusions. Steve Albini, producer of Nirvana and Pixies albums and himself a touring musician, says in this keynote that the Internet has solved many problems for musicians and audiences alike and that things are actually better now than before.

Finance got networked in the wrong way

As an example of the destruction being wrought by the Siren Servers, Lanier asserts that the Great Recession was in part caused by their existence in the financial sector.

"Consider the expansion of the financial sector prior to the Great Recession. It's not as if that sector was accomplishing any more than it ever had. If it's product is to manage risk, it clearly did a terrible job. It expanded purely because of its top positions on networks. Moral hazard has never met a more efficient amplifier than a digital network. The more influential digital networks become, the more potential moral hazard we'll see, unless we change the architecture."

It's clear that Lanier believes that this problem can be solved by reconfiguring and improving the architecture of our digital networks.

Siren Servers and the rise of surveillance as a business model

As has become increasingly clear in the past few years, the dominant business model in Silicon Valley is one based upon surveillance.

Surveillance is the mechanism by which the Siren Servers gather much of the data they analyse and profit from.

Surveillance cameras

According to Lanier, many Silicon Valley techno-libertarians handwave any concerns about corporate surveillance.

"Surveillance by the technical few on the less technical many can be tolerated for now because of hopes for an endgame on which everything will become transparent to everyone. Network entrepeneurs and cyber-activists alike seem to imagine that today's elite network servers in positions of information supremacy will eventually become eternally benign or just dissolve."

However, what good reason is there for owners of these immensely powerful and valuable Siren Servers to one day willingly become transparent and open up in order for this power imbalance to be resolved?

On the supposed emergence of some kind of sharing/socialist utopia arising out of the aggressive right-wing libertarian practices of Silicon Valley, Lanier mocks:

"Free Google tools and free Twitter are leading to a world where everything is free because people share, but isn't it great that we can corner billions of dollars by gathering data no on else has?" If everything will be free, why are we trying to corner anything? Are our fortunes only temporary? Will they become moot when we're done?"

These "network servers in positions of information supremacy" hide their code and algorithms behind patents, IP-laws and copyright.

A whole legal facade has been erected precisely to prevent the hopeful scenario of "elite network servers" dissolving into an utopian abundance.

Lanier's proposed solutions

Cyber-keynesianism

Lanier describes his proposed solution as a sort of Cyber-Keynesianism. Based upon the idea from the economist J.M. Keynes that "stimulus" might be able to kick an economy out of a rut.

Lanier provides a qualitative explanation of the stimulus theory which I haven't heard before.

On a multi-dimensional mathematical landscape (as could be plotted based on the parameters of an economy), there are peaks and valleys. Let's assume that peaks are "optimal" economical configurations, where the greater good is best served, and valleys are horror scenarios (depressions, recession, collapse etc.).

Not all peaks are of equal hight, and when you are on a peak, you are usually surrounded by valleys. There might be a higher peak (i.e. better configuration) available, but it's not clear how to get there, since moving there might mean moving through valleys.

The idea behind stimulus is to provide the impetus, a kick if you will, to make this transition to a higher peak.

Technical Solution

Laniers solution on a technical level is to use a network with bidirectional links, instead of unidirectional links as we currently have with the web. The architecture of this network is inspired by Project Xanadu already founded by Ted Nelson in 1960.

Whenever someone publishes something on the web they would be informed if someone else links to their work, due to the bidirectionality of the links. This would supposedly put a stop to copyright infringement and also allow content creators to correctly identify and bill the consumers of their media.

Of course, this would only work if there were no anonymity on the web.

Lanier further proposes a single, public, digitally networked marketplace where every participant is tied to his personal real-life identity, and where anything anyone creates online has to be paid for.


My Criticism

I thoroughly enjoyed reading Lanier's analyses, and his descriptions of the mechanisms behind Siren Servers in particular. This was for me the best part of the book.

However, he often hand-waves obvious counter arguments or advances his own arguments in a non-rigorous way, often relying on personal anecdotes.

Digital networks' supposed role in hollowing out of the middle class

Considering his central thesis, that digital networking is hollowing out the middle class in developed societies, I'm not sure how much blame for that can be assigned to digital networking. Other plausible factors include demographic change, globalisation and the money printing policies of Central Banks which are fueling asset price bubbles while real wages remain stagnant, thereby making wage-earners poorer.

My expectation is that no particular factor can be singled out, and that the causes are multiple, varied and intertwined in complex and difficult to understand ways, precisely because the economy is a complex, chaotic system (in a mathematical sense).

Lanier fails to meaningfully address Free and Open Source software

Lanier only very superficially addresses the role of free and open source software (FOSS) and doesn't even mention (let alone extensively cover or deconstruct) the ideology of Free Software at all.

As a quick recap to people who don't know the difference between "Free Software" and "Open Source software":

Both groups concern themselves with creating software where the source code is visible, as opposed to software where you cannot inspect the source code, and are therefore unable to modify or know what it is really doing.

Gnu and Linux

The GNU and Linux mascots. The GNU represents Free Software

"Free Software" (as in freedom, not price) however stresses a moral and ethical dimension of software development and usage. It basically boils down to the fact that non-free software creates a dangerous and immoral power imbalance, slanted in favor of the creator of the non-free software and against the user of that software. [2]

Open Source software on the other hand ignores the moral dimension completely and takes a much more expedient approach. It simply asserts that developing software in the "open", in other words, with the source exposed, is a superior form of software development which will result in better code.

In the index of this book, there is no entry for "Free Software" or "Open Source Software". There is one entry "open source applications".

Lanier doesn't even attempt to address the viewpoint of the Free Software movement, which has been warning about these issues (structural power imbalance, Siren Servers, ubiquitous online surveillance) for decades. This is in my opinion a glaring omission from the book.

Lanier might have coined the catchy phrase Siren Servers, but his analysis of their role and negative effects on users and society might have well come from someone from the Free Software movement. See for example the article Who does that server really serve?.

As the saying goes, "There is no cloud, only other people's computers" [3]

Lanier of course, comes to a completely different conclusion on what needs to be done to resolve the situation, and appears to hold proponents of Free and Open Source Software (dimissively calling them "racals", "openness crusaders" and Pirate Party types), not only in contempt, but partly responsible for the emergence of Siren Servers and rise of surveillance as business model.

It is clear that if Lanier were to meaningfully address all ideological opponents of his vision for the future, then he would have had to address the ideas behind Free Software in a methodical and rigorous manner and not with dismissive hand-waving.

The fact of the matter is that Free Software does provide an alternative solution to the current mess of online surveillance, increasing risk and fragilization, extreme power imbalances and centralization into Siren Servers.

The solution is to ensure that the software you use is free. If software being used by a service is free and issued under a license that ensures its freeness, then that server will not have the ability to eventually become a Siren Server.

The reason for this is that users will know exactly what the software does with their data and will have the ability to switch to a different provider (while taking all their data with them), or to host it themselves, at a moment's notice.

This means that the power balance is no longer slanted in favor of the service provider, and users are much more in control of their data and content.

Lanier's hypocrisy

Throughout the book, Lanier criticises in particular Google and Facebook for their creepy surveillance business models and their usage of Siren Servers.

Lanier himself works for Microsoft, a company that is feverishly competing with Google and Facebook in surveilling users and in establishing their own Siren Servers.

However, Lanier of course does not at all address this conflict of interest or that fact that his employer is just as complicit as the companies he calls out. This apparent hypocrisy severely detracts from the integrity of his book.

Lanier's superficial and misconstrued criticisms of "openness"

Lanier's refusal to meaningfully engage with the different ideas and philosophies behind what he calls "openness" is for me one of the most disappointing aspects of the book.

Wikipedia

He deceptively equates Wikipedia and Facebook, saying that both are a culmination of the "information should be free" meme and that both contribute to the devaluation of user created content. In a previous book, he even calls Wikipedia "Digital Maoism".

Wikipedia is concerned with creating a digital Commons. By voluntarily contributing to the commons (as Wikipedia contributors do), you are not devaluing your work, you are making it priceless, in both the literal and figurative senses of the word.

Conversely, the information on Facebook is not free and is not part of a commons.

I find it good to know that there are some things in this world on which one cannot put a price tag, despite the efforts of people like Lanier.

As the saying goes. "A cynic is someone who knows the price of everything and the value of nothing."

Wikileaks

Lanier incorrectly claims that Wikileaks is concerned with abolishing all privacy. Anyone who has made even a half-arsed effort to understand what Wikileaks is about would know that they stand for transparency of the powerful and their institutions, in order to keep them democratically accountable, and that they don't advocate the elimination of all personal privacy.

Of course, this is a common mischaracterization of this organization, often done to intentionally discredit them. Either Lanier is also trying to discredit them or he couldn't be bothered to properly research their stated goals.

Linux

Lanier claims: "A linux always begets a Google" as if it's is somehow responsible for the emergence of Siren Servers. This doesn't explain Siren Servers built upon proprietary software, such as StackOverflow which uses Microsoft.

Inasmuch as one could argue that Siren Servers are the result of open source software (note the distinction) it is exactly due to open source's gutting of the moral aspect out of free software.

Lanier wants to remove anonymity on the web

Lanier criticises tech companies for their creepy surveillance business models, and yet his proposal for "fixing" the internet would eradicate all privacy online and have all human interactions facilitated via a single, public market place. It would be a surveillance and privacy nightmare.

Concerning anonymity on the web, Lanier says this is the result of the "pot-smoking liberals" and "paranoid conservatives" who originally built the system and who thought anonymity was "cool". Such a perfect example of his dismissive, non-rigorous and arbitrary reductionism which completely fails to meaningfully confront the potential benefits or dangers of anonymity on the web.

Lanier completely fails to address the fact that anonymity is often the only defence people may have against corrupt and malevolent state power. Whether this applies to activists during revolutions or to conscientous whistleblowers, doing away with privacy would be throwing away one of the few ways of ensuring free speech online.

Tahrir Square on November 27, 2012

Tahrir Square protests in 2012.

Monetizing all human interactions debases them

By wanting to turn the web into a giant marketplace where everyone operates with an exposed identity and every digital creation, no matter how trivial, needs to be paid for, Lanier wants to further the already ongoing monetization of the public commons. As Charles Eisenstein writes sacred economics, monetizing all human interactions actually debases them.

Lanier calls his approach "humanitarian"; it is anything but. Instead, by monetizing and thereby commodifying every online human interaction, whether it's a joke, a compliment or a statement of support, these interactions are stripped of their inherent sacred quality of humanity.

If you know that someone who sends you a message of support or love will receive a micropayment for that message, could the original intent of the message not be called into doubt?

My proposed solutions

It's about freedom, not cost

Lanier's proposed solution ultimately has to do with costs and trying to find a way to get people paid for anything they write and then publish online, no matter how trivial.

However, in so doing he comes up with a proposed solution that does away with all privacy and anonymity.

His proposal of using a network with bidirectional links might threaten the Siren Server status of services such as Facebook and Google. However, it's not clear how such a network will solve the problem of Uber or AirBNB becoming Siren Servers, thereby pushing out the middle class jobs of taxi drivers and guest house operators.

It's not only about the network architecture, as Lanier suggests. What's more important is Software and Computing Freedom.

If you don't have the freedom to study, use and modify the software that you use, then you will be systematically exploited and preyed upon, no matter what the network architecture looks like.

Free software specifically, is defined by these four freedoms:

  • The freedom to run the program as you wish, for any purpose
  • The freedom to read the source code and to modify it as you wish.
  • The freedom to redistribute original copies so you can help your neighbor.
  • The freedom to distribute your own modified copies to others.

Being able to read the source code is essential for software to be free. An organisation is however only compelled to share their source code if they intend to distribute compiled copies as well. If the software is used privately, i.e. in-house, then they are under no such obligation.

Open Source on the other hand, also gives the right to read the source code, but many open source licenses do not ensure the last two freedoms, and therefore offer weaker protection.

There is however a large overlap between the two groups. Most open source software is also free software, and vice versa.

Free software coupled with free and open protocols means that users are not beholden to specific service providers. It's exactly this aspect of it which is why the dominant tech companies avoid free software like the plague and instead embrace the watered down notion of "open source". Their goal is to maintain strict vendor lock-in, making it difficult for users to switch to other service providers by holding their data and online personas ransom. Something which they could not do if their software respected the above four freedoms.

Who pays for fee software?

When discussing free software, the question often arises how one should make money with software if you cannot enforce artificial scarcity, vendor lock-in, holding user's data ransom or spying on them.

This is perhaps a topic for another blog post. In short, it definitely is possible to make money writing freedom respecting software and the more people insist on using only free software, the more economic opportunities will arise.

Software is better seen as a living, changing and evolving ecosystem, than as a static object. It needs constant care and attention, to keep it relevant, accurate, applicable and functioning. This care and attention is a service which can and should be paid for.

Who pays for this service? The users do. Whether they are governments, universities, corporations or private individuals.

How do they pay for it? Organisations sign service and support contracts. They also commission the writing of new features or entirely new applications.

Private individuals can pay for free software through crowdfunding, bounties, and donations. [4]

Public institutions and foundations can issue grants or fund the development of free software for usage in government and publically owned enterprises.

If all governments decided to only fund and use free software, it would create a massive amount of investment in the sector. Additionally, money spent on developing free software can be invested in a country's own programmer workforce, instead of it being sent overseas to oversized foreign corporations.

However, those companies that currently spy on us, hold our data ransom and restrict our digital freedoms, won't change by themselves. It's up to the users to take matters into their own hands, to demand freedom and to support those companies, organizations and people who develop free software.

Decentralization

The problems Lanier identify are largely caused by data-assymetry (who has data on whom) and the resulting siren servers.

An architectural approach to solving this problem, which complements free software, is decentralization.

Decentralization refers to a network structure (called topology) where each node in the network can connect directly with any other node, without the intervention of a middle man (such as Facebook or Google).

In the network topologies pictured here, the fully connected topology is the most decentralized, while the star network is the most centralized.

Network Topologies

Network Topologies

Creating large decentralized networks is more difficult than creating centralized ones, which is why the web has come to rely on centralized Siren Servers to such a large degree.

However, much exciting work is being done on decentralizing the web and putting people back in control.

Consider Finance, and Lanier's assertion that Siren Servers contributed to the financial crisis preceding the Great Recession.

It's more difficult to have a financial Siren Server when financial data is decentralized and therefore effectively available to everyone.

This is already the case with decentralized cryptocurrencies such as Bitcoin. The bitcoin protocol is an open and free protocol, the original bitcoin wallet is free software and the transaction information contained in the blockchain is available to all.

An improvement upon Bitcoin would be to allow truly anonymous transactions, and this appears to be on the horizon.

Bitcoin and cryptocurrencies are however only one application which can be built on top of the revolutionary distributed consensus blockchain technology upon which it rests.

There's already work on distributed cloud hosting, distributed microblogging and a myriad of other services running in a decentralized (non-Siren Server) manner.

Federation

An older idea than blockchain-based decentralization is federation. Think of the way in which you can send an email from a Gmail account to someone with a Yahoo account, or to someone who hosts their own mail server.

Why can we send email to people with different service providers, but we cannot send chat messages from a Google account to an iMessage account, or between Whatsapp and Yahoo messenger?

This ability of email is the result of something called federation. The ability of servers from different providers and with different implementations, to communicate and relay messages between one another.

Email is federated but facebook messages are not, because you cannot send a facebook message to someone outside of facebook.

There is no technical reason why instant messaging cannot be federated. In fact, XMPP, the protocol on which many of the messaging services are originally based, allows federation.

The reason we don't have federation in messaging is because it's not in the interest of Siren Servers. It would actively undermine their dominance.

If any of the big email providers, Google, Yahoo, Microsoft or Apple, could get away with only allowing emails within their own system, they would have done it by now. The proof is in the fact that they actively prevent federation in their messaging apps. The reason they cannot do it with email is because federated email achieved critical mass before any single company could kill it off.

Unfortunately, this was not the case with instant messaging. Therefore, if you actively want to work against Siren Servers, you should only use instant messaging servers which support federation, such as XMPP.

You can sign up for a free XMPP chat account here. And if you'd like to chat with me, add me as a contact: jc@opkode.com

Conclusion

As you can see, I believe in a completely different remedy than the one which Lanier suggests. One that's based on a methodology, legal framework and software which is already being used instead of something which Wired magazine in 1995 called the longest-running vaporware project in the history of computing.

I thoroughly enjoyed Lanier's analyses and his unique perspective on things which I think made the book worth reading.

However, his non-rigorous approach, his dismissiveness or ignorance of alternative narratives and approaches to resolving the issues he mentions, as well as his hypocritical omission of his own employer's complicity, left me disappointed and uninspired.

Footnotes:

[1]Bruce Sterling calls them "The Stacks", vertically integrated social media. See Bruce Sterling At SXSW 2013: The Best Quotes.
[2]If you're interested in more detail about Free Software, watch this presentation by Richard Stallman:

Introduction to Free Software and the Liberation of Cyberspace

[3]The earliest reference to this that I could find, is this article
[4]See Kickstarter, IndieGogo, Startjoin, Patreon, Bountysource, Gratipay and Snowdrift.
Read and Post Comments

Antifragile Software Ecosystems

| categories: open-source, bitcoin, cryptocurrencies, antifragile, foss, plone | View Comments

Antifragile, by  Nassim Nicholas Taleb

I recently finished reading the book Antifragile, by Nassim N. Taleb, in which he discusses the idea of "antifragility", the property that allows something to ultimately improve and grow stronger when subjected to volatility and stressors.

I highly recommend Taleb's books, they have a tendency to throw conventional wisdom on its head and make you think differently of our world, in which randomness, volatility, opacity and non-linearity takes precedence.

As a software developer, I kept wondering how the insights gained from this book apply to the world of software. Especially given the fact that software is very fragile. The answer lies in the communities or ecosystems that form around software. Many of them are fragile, but some are antifragile.

If you're also a software developer, you'll want to be in the second category.

What is antifragile?

Taleb splits everything into three categories, fragile, robust and antifragile.

The initial chapters explain in detail why robust is not the opposite of fragile, and why the neologism "antifragile" is needed to describe the opposite of fragility.

Medusa is antifragile

Medusa, courtesy of Wikimedia.org

If fragile systems are sensitive to stress and volatility, and need to be kept "safe" from harm, lest they suffer or break down, then antifragile systems are ones which ultimately benefit from randomness and stress.

In Greek mythological terms, one can think of Medusa, which grows another two snakes on her head for each one that's cut off. That's antifragile.

So fragile and antifragile can be considered opposites, e.g. positive and negative, while robust is simply neutral. It suffers little harm from volatility but also gets little to no benefit.

Evolution is antifragile

Ostriches have evolved from dinosaurs

Ostriches have evolved from dinosaurs

Some good examples of antifragile systems are found in nature and ecology. For evolution to work (i.e. for a species to adapt to its randomly changing environments), random mutations are required between individuals in that species.

A random mutation in a member of a species holds the possibility of making that member better adapted to its environment. By virtue of it being better adapted, that member will on average live longer and produce more offspring, which will be likely to have inherited the same advantageous mutation. The process continues, propagating the beneficial mutation throughout the species, in so doing, the species evolves and becomes more fit and adapted to its environment.

Nature is able to discard less well-adapted specimens and keep the better-adapted ones. An ability which Taleb refers to as optionality. It has the option to keep the "good" specimens and discard the "bad" ones.

The opposite of an option is an obligation. Nature is under no obligation to keep the poorly adapted members of a species, and if it were, it wouldn't be antifragile anymore.

So, although individual members within a collection (like a species) are fragile to certain stressors, the overall effect on the collective level is antifragility.

Software is fragile

/img/fragile.jpg

Image courtesy of Wikimedia.org

Unmaintained and unsupported software degrades when exposed to the stressors of time and use.

The more software you install on your computer, the higher the chance that your machine will eventually become slow and prone to crashes and bugs.

The word "bitrot" is sometimes used to refer to the gradual degradation of a software program over time and is related to the idea of entropy.

Large software systems that are composed of multiple interacting subsystems, are especially vulnerable to bitrot, due to the fact that a given subsystem cannot anticipate all the bugs or changes that might be introduced to other subsystems over time.

The more a piece of software is used, the more variance there is in how it's being used. For example in the order, type and amount of inputs it receives as well as in the hardware its being installed on.

Lets assume that a very specific interaction with the software, which the programmer never foresaw, will cause the software program to crash and become permanently degraded.

Given enough time, someone will eventually interact with the software in this specific way, thereby permanently degrading the functionality of the software program (for example due to data corruption).

Are software businesses also fragile?

Businesses are generally fragile. They require a certain level of stability to operate and are sensitive to volatility (like regulatory changes, lawsuites or changing consumer tastes).

An antifragile software business however would be one that does well whenever some metric related to the software experiences increased volatility.

Remember:

  • something that's antifragile wants more volatility, because the benefit from increased volatility outweighs the harm.
  • Similarly, you are fragile to a specific source of volatility when the potential downsides outweigh any upsides.

So, let's take "number of security flaws" as the metric.

The fortunes of a company that does security audits would improve every time there is a publicised security flaw in the software itself, or the software is compromized by an attacker.

They would therefore be antifragile to the "bad news" of exposed security flaws.

Perhaps with an upper bound. If there are too many security flaws, the reputation of the software itself might suffer irrevocably, and with it that of the company.

This illustrates the point that antifragile systems can handle volatility and stressors up to a point. Given enough volatility and stress, the entire planet can die out.

Conversely, a company whose business model depends on the software being stable, and on attacks not happening, would be fragile to the software having security flaws.

A community of fragile members can itself be antifragile

So, as mentioned earlier, an ecosystem can be antifragile, even if the individual units that make up that system are fragile.

It is however important that the units themselves are relatively redundant and that the fragility is not systemic inside the entire ecosystem.

To put it in more technical terms, units inside the system need to be "hot-swappable" and their interactions with one another should be "loosely coupled". This is necessary to prevent complex and hard dependencies within the system, which in the case of breakdown of one unit would cause cascading failures and damage rippling through the entire system.

Antifragile software ecosystems

So, we now get to the crux of this article.

Software itself might be fragile, as well as businesses built around that software, but the larger ecosystem that encompasses the software and businesses, can be antifragile.

A software ecosystem that is distributed over many small and relatively self-contained businesses, each providing services around a common shared resource of Free and Open Source software (FOSS), coupled with a low barrier to entry for new businesses, is antifragile.

Conversely, a software ecosystem that is centralized around one large provider (often playing the role of gatekeeper or rent-seeker), artificially restricting access to the software, attempting to stifle any competition with its own services, is fragile and sensitive to large negative Black Swans (difficult to predict but extreme events).

I'll elaborate on why this is by identifying the requirements for an antifragile software ecosystem.

Requirements for an antifragile software ecosystem

  • Distributization

    Centralized systems provide a large single point of failure that can cause the collapse of the entire ecosystem. Distributed systems can take shocks much better and are able to route around failures and breakdowns.

  • Redundancy

    When one unit breaks down, others need to be able to pick up the slack. For that to happen, they need to collectively be able to perform the functions of the defunct one. For example, if a software community relied solely on one unit in that community for supplying hosting services, then the entire community would break down if that one unit collapsed.

  • Optionality

    The individual members in an antifragile ecosystem need to be able to modify and adapt.

    For a software ecosystem, this means the ability to modify and adapt the software itself. Only through being able to modify the software in a distributed, bottom-up way, can the ecosystem withstand stressors in an antifragile way, by making use of optionality.

    Optionality refers to having the option to take (or do) something or not. As mentioned earlier, it's how evolution is able to "design" extremely sophisticated organisms, by keeping the "good" ones and discarding the bad ones, and it applies to software ecosystems as well.

    The Plone logo

    For example, in the Plone CMS (content management sytem) community, add-on modules are developed in a redundant and distributed way. There are multiple blog add-ons, commenting add-ons, newsletter add-ons and so on. Through a process of optionality and selection, the good ones rise to the top while the weaker ones are eventually discarded.

Why the requirement for Free and Open Source software?

FOSS facilitates all three of the above requirements. Communities and ecosystems that develop around FOSS are by nature distributed, redundant and provide optionality.

Distributization and redundancy come from the lack of gatekeepers, "rights-holders" and rent-seekers. Anyone can start using FOSS and selling services around it from anywhere in the world. No license, special qualification or permission required.

Optionality comes from the fact that anyone can do trial-and-error tinkering with the software and keep their improvements. They can also choose the best or most appropriate changes and modifications made by other people.

Without optionality a software ecosystem will not be antifragile, and without FOSS, a software ecosystem will not have optionality.

Additionally, software that is Free (as in Libre) displays qualities much more akin to information than non-Free software, which due to it's source opacity functions more as a black-box application. As Taleb explains in his book, information is antifragile. You can't easily kill it, and by trying to suppress it you have to acknowledge its existence and refer to it.

Lastly, the lower barrier to entry fosters competition.

In order to differentiate themselves, individual developers or businesses need to compete on quality and/or price. It's survival of the fittest, as opposed to the rent-seeking, vendor lock-in and monopoly so often seen in proprietary software ecosystems.

Note

In the book, Taleb alludes to Silicon Valley's "fail fast, be foolish" model as being antifragile as opposed to a more centralized environment with inherent systemic risks, such as the New York banking system (his example).

Silicon Valley might provide an antifragile method for producing businesses, but the software produced by these businesses don't necessarily result in antifragile ecosystems.

Look at the software businesses that have arisen out of Sillicon Valley. Facebook and Google are both large and centralized. Increasing centralization and size results in increasing fragility of the web.

If Google's servers were to go down (or be compromized or blocked by a government), a very large part of the web would cease to function. The chance of Google's servers going down are of course very small, it's a negative Black Swan event. Something which society nevertheless needs to shield itself against.

Antifragile against what?

Having mapped out the requirements for an antifragile software ecosystem, it makes sense to consider the kinds of stressors such an ecosystem will be antifragile against.

This section is a bit more speculative, mostly because we cannot know the exact nature and extend of the stressors a system will be exposed to. Luckily we don't need to know the stressors to be able to make use of antifragility.

It's also important to know that stressors are information. In fact, stressors often appear through the disclosure of "negative" information. When negative information is revealed, it can cause shocks to software ecosystems. The fragile ones will suffer and might even break down, while the antifragile ones will ultimately benefit.

Here are some examples of stressors a software ecosystem can be antifragile against:

  • Malicious Hacking (Cracking)

    FOSS is more resistant to cracking than closed source software. Additionally, distributed systems are much harder to target and infect.

    Increased awareness and fear of cracking, makes FOSS, its ecosystems and communities more attractive to software users (individuals, companies, governments etc.) and therefore antifragile to the "bad" news.

Surveillance cameras

Image courtesy of Wikimedia.org

  • Surveillance

    A big part of surveillance is actually also cracking, so the same rules apply here.

    It's much more difficult for eavesdroppers to embed backdoors in software with freely available and visible source code.

    Encryption software is especially vulnerable and can't be trusted if its closed source.

    News of security flaws spread quickly throughout the community and fixes are generally pushed out almost immediately. You won't easily have the problem of a FOSS community member privately disclosing security flaws to an intelligence agency (so they might use if for surveillance) while hiding it from the users.

    Again as surveillance and privacy fears increase, FOSS becomes more attractive, making the communities and ecosystems around it more antifragile.

  • Negative economic circumstances

    Many governments are experiencing budgetary constraints and employing austerity. Governments are able to make significant savings when they switch to FOSS ecosytems. Not mentioning the increased autonomy and control of their software and resulting digital formats.

There are of course many more stressors that might ultimately benefit distributed FOSS ecosystems, for example censorship and economic boicot, but for the sake of brevity I won't go into detail for each one.

Conclusion

I've mapped out the requirements for a software ecosystem (and community) to be antifragile, and have alluded to Plone as an example of such an ecosystem.

The Bitcoin logo

The Plone software, like other Free software (Drupal, Wordpress, Apache, Firefox, Libre Office, GNU/Linux etc.), is a technological commons. It belongs to humanity as a whole.

Everything I wrote also applies to the ecosystems around cryptocurrencies, like Bitcoin, Litecoin and Anoncoin.

There are usually vibrant communities of developers, packagers, integrators, evangelists and users around these pieces of Free software, and they usually conform to the requirements mentioned above. Members are distributed all over the world, there is redundancy in development skills and support offerings and the source code is Free (as in beer and in speech).

Thanks for reading and please leave a comment if you have anything to add, I'd be happy to discuss the finer details perhaps missed in this article.

Read and Post Comments

Why you should open source your python eggs.

| categories: python, pypi, open-source | View Comments

This blog post is based upon my experience in developing add-ons for the Plone open source content management system. It's therefore mostly aimed at the Plone community, although some of the insights mentioned might also apply to other communities and to web-development in general.

In the last 15 months, I have released 15 Plone add-on products as open-source python eggs, available for download on Plone.org and Pypi, the python package repository. In the preceding 4 years of Plone development, I released only a single product.

/img/python_egg.jpg

Image by MrTopf

In this process, I have witnessed a personal metamorphosis. I can draw a relatively clear line between 'Ye Olde Worlde' web-developer who was set in his ways and narrow minded in his methods, and the new! improved! open source developer.

I've seen the transformative power of (the act of) open sourcing and although I'm sure that a lot of what I have learned is already old hat to many developers, I would like to share the insights I have gained from this experience. Many developers have been made aware of the inherent advantages of collaborating in an open-source community, but to take part in it yourself and to experience the inherent benefits first hand, is the only way of developing a true conviction for the appropriateness of open source collaboration.

The fact is, that you have a lot to gain by engaging with the community, through collaborating, sharing your ideas and making your code public.

Open source everything that might be useful to others

It's relatively common practice in the Plone community to develop a so-called 'policy package' for a new project. The policy package should tie everything together, configure the Plone application to the client's needs, and install all the other add-ons.

Additionally, a theme egg for implementing a custom theme is usually also required. These two packages, together with a collection of third-party and in-house developed packages (each responsible for a specific feature set), on top of Plone itself, constitutes the software product presented to the client.

Before I was introduced to the idea of a policy egg by Martin Aspeli and others who championed the idea, I would usually when given an assignment, develop only one Plone add-on product for it. This add-on then contained all the custom software code that I wrote for this specific client and/or product, regardless of whether certain parts of it would also be beneficial to other people or projects.

This is of course not the proper way to develop a sustainable, reusable and configurable solution for a client. Instead, every piece of functionality that constitutes a logical grouping of features and that could theoretically be reused somewhere else, should be packaged separately in its own python egg.

And here's the catch: you should ideally open-source everything except the policy package. Even the theme egg should be considered for open sourcing.

The policy package serves as the puppetmaster, that ties these different pieces together to make the integrated unit (or puppet show). It is the proprietary configuration that contains the client specific implementation details, the client's copyrighted logo and images, the site copy and everything else that makes the site unique to the client. It's the secret sauce or the kernel's 11 herbs and spices, in a matter of speaking.

With regards to the rest of the custom developed eggs in your project, release them as open source eggs on Pypi! You have a lot to gain by doing so.

Let me elaborate on why...

Advantages

Open sourcing your code forces you to be a better programmer

The implicit mental adjustment coming from the knowledge that you are writing source code which will be read and modified by others, is already enough to considerably enhance the quality of the code that you will write.

Why? Because of the added social pressure.

Glossophobia, the fear of public speaking, rumoured to be the most common fear in the minds of people, is a consequence of the fear of public humiliation.

Nobody wants to be humiliated publicly. Not for being a poor speaker and not for writing bad source code. The fact is, that you will make more of an effort to write clear and legible code that follows best practices when you are writing for an audience (the community of developers) than when you are writing for personal or 'in-house' consumption.

Before I started writing code with the intention of open-sourcing it, I would be much more inclined to include quick and dirty hacks to "save time" or to "simplify" things. Ironically, quick hacks such as hardcoding a value or overriding a template instead of putting it in a viewlet will in the long run, waste time and overcomplicate things.

A quick real world example: Your boss or client wants a facebook style action panel on the bottom of a site.

Do you:

  • Decide not to open source this functionality, and instead add the styles and templates to the theme egg while hardcoding the links on the panel because it's quick and easy?
  • Decide from the getgo that you will open source this and therefore create a separate egg for this piece of functionality, that adheres to best practices to make it reusable, extendable and configurable?

The second option might take longer initially, but its benefits pay off in the long run, especially when the client requires technical support.

The community provides an additional level of quality control

I'm careful here to add the word 'additional' since you should not solely rely on the community to test your code for you. On the contrary, other people will be reluctant to try out your products if you don't put in the initial effort or write tests and doing quality control yourself.

That being said, granted that you have released a quality package, you will receive an additional layer of community powered quality control by virtue of the fact that the developers and integrators using your code will let you know when they run into bugs, and often even fix them for you! "With many eyes, all bugs are shallow."

The community provides new perspectives, ideas and enhancements.

This past year, various developers have contacted me with suggestions, ideas and improvements to some of the packages that I have released. Some have made a meaningful amount of effort to explain why a certain solution might be better, and in the process have provided me with a free education and broadened the scope of my Plone dev knowledge.

Overall, the quality of the packages that I have released, has been improved through the sustained contributions of other developers in the Plone community. This is after all one of the main advantages of open source, seen from a purely pragmatic point of view. To witness this process in action and with code that you yourself have contributed, is very satisfying.

Social currency (A social C.V or Résumé)

Curriculum Vitae

Photo by Superbez

Open source contributions, being easily searchable and in the public domain, form an authorative record of the depth and quality of an individual or company's output.

Compared to the sections traditionally contained in a Curriculum Vitae (Résumé), your open source contributions form your work experience section, while their contributors and users serve as references. Sustained open source contributions coupled with helpful and courteous interaction with the community, provides a more authoritive record of ability and character than any CV ever could.

Suggestions:

I'd like to round up this post by making some suggestions which I consider will better the chances of your product not only being used by the community, but also receiving code improvements and bugfixes.

Don't host in-house:

Use the collective repository to host your Plone source code. We're lucky to have it, and that's what it's there for. Alternatively, use Google's project hosting or github. Although, in my opinion, it doesn't make much sense to call something collective.* if it's not hosted on the collective repo (I know others disagree with this).

You want the barrier to entry for participation in your code to be as low as possible. That means making it as easy as possible to quickly provide improvements and bugfixes.

If you host your open source products privately, you might gain users, but you will have far fewer contributors willing to fix and enhance your code, simply because they're not willing to jump through hoops to be able to do so. Private or walled off hosting discourages contributions and promotes forking. Which would be a forking shame (sorry, I had to).

Start your project in the open:

Don't develop your packages privately, waiting for them to be "good enough" to be added the to a public repository.

Start from the beginning on developing publicly on a repository such as the collective. This will ensure that you stay aware of the fact that you are writing code for others and not just for yourself. As mentioned before, when you consider the fact that others will have to read and digest your code, you will more carefully consider your method, class and variable names. Do they make sense? Are they consistent? Is the code explicit and readable? etc.

Release your Plone eggs on both Pypi and plone.org:

Some users prefer to search for Plone add-ons via plone.org, while others use Pypi. Cater to both to maximise your visibility. Plone.org goes a step further by among other things allowing you to upload a screenshot and your company's logo.

Tools such as zest.releaser or jarn.mkrelease make releasing eggs a breeze. Use them! :)

Conclusion:

When it comes to open sourcing your code, the fear might arise, that by making your code available for others to use and adapt, you might in some way lower your chances of monetizing the code. For people living with a mindset of scarcity, in which ideas and creations must be guarded closely to safeguard their success and to prevent others from "stealing" or profiteering from them, it might be difficult to make the paradigm shift necessary, towards a mindset of abundance, opportunity, sharing and collaboration.

The fact is, that you have a lot to gain by engaging with the community, through collaborating, sharing your ideas and making your code public. ;)

Read and Post Comments