The Proprietary Renaissance

noun, re·nais·sance often attributive
\ˌre-nə-ˈsän(t)s, -ˈzän(t)s, -ˈsäⁿs, -ˈzäⁿs, ˈre-nə-ˌ,
chiefly British ri-ˈnā-sən(t)s\

: a situation or period of time when there is a new interest in
something that has not been popular in a long time

: a period of new growth or activity

- Merriam-Webster

We are entering a Proprietary Software Renaissance.

It didn’t happen overnight. Hackers didn’t turn on their sticker-covered laptops to find them suddenly running the latest proprietary release from Redmond. GNU/Linux wasn’t outlawed by the NSA. Mailing list flames are as strong as ever.

But somewhere in the past several years we started living in a future where the future success of proprietary systems seems a lot brighter than those of Free Software/Open Source[note]”Free Software” was always a bit of scary label to use around companies despite the fact that Free-as-in-Freedom does not have to equal Free-as-in-Beer, so the term “Open Source” was developed to explain the idea of having software with the source code available under some kind of share-alike and modification-allowed license. But whilst software can both be Free Software and Open Source software, there are differences. The major difference is that Open Source lowers the importance of Freedom and instead focuses on producing technically high quality software in a collaborative fashion. Because I’m more concerned about Freedoms in this post, I’m using the term Free Software instead of Open Source[/note] platforms.

This isn’t saying that Free Software isn’t successful. It’s been immensely successful, without Free platforms like BSD, GNU and Linux our current world would look quite different.

Certainly the success and development of the internet and modern technology companies have been driven predominately by these Free platforms and enabled them to break past some of the existing proprietary gate keepers in the process. And it’s not limited to new-age companies, even the traditional corporates have adopted Free Software in their businesses. RedHat claim 90% of Fortune 500 companies rely on Red Hat’s open source products [note] It’s important to note that commercial success of Free Software comes more under the label of Open Source software which is marketed more as providing an escape from vendor lock-in and granting the ability to customise for your business, rather than the pro-community, pro-freedom aspects of Free Software.[/note]

Because of the freedom to innovate, Free Software has succeeded amazingly well in the technology sector as businesses adopt components to make their products more reliable or more competitive.

 

But it’s failed with average users

Not due to bad technology. Not due to lack of proponents pushing awareness of Free Software. The awareness of Free Software is out there and advocacy is still strong. Even the US government has been helping raise awareness of the importance of Free Software lately.

But Freedoms mean little to those whom have them until they’re lost. Many people won’t ever exercise their right to protest, but as soon as that right is removed, that right becomes more important than life itself.

Users don’t see how Free Software has given them the free and open internet that they currently have rather than captive portals controlled by a select few large American corporations. Users don’t see how the cost and quality of their favourite device has been improved thanks to Free Software. Users don’t see how bad the situation would have been if we were stuck in a world where a single vendor had majority control.

For a long time there was the belief that the success of the GNU/Linux platform in the server space would start to translate to the desktop sector and we would see GNU/Linux systems on sale from mainstream vendors.

But this never really eventuated. A few companies, Dell included, toyed with very select GNU/Linux-enabled product ranges, but never offered it in a serious way across their fleet.

Part of the problem was that despite it’s zero-cost licensing, the choice of GNU/Linux made computers more expensive for the vendors since they could no longer profit from the much-hated bloatware they preload the computers with which subsidised the licensing costs from Microsoft[note]Higher support costs would also be likely, but this would have resolved itself with time if the market share grew to a level where the

And consumers voted with their wallets – Windows was “free”[note]As in really bad beer[/note] when buying an OEM machine due to the bloatware subsidies making it cheaper than GNU/Linux, ran more consumer applications and games and was a more recognised brand. Sure it restricted your freedoms and was technologically inferior but if you just cared about running apps and not about freedom, it was a more logical choice.

Sadly consumers don’t see an Free Software device and think “this is a better quality device since it respects more of my freedoms“, they look at the cost of the device and the apps available for it and make a decision accordingly.

And at the end of the day, most users[note]Readers of this blog probably excluded.[/note] aren’t interested in running an operating system. They want to run applications that enable them to get something done and there’s no difference for them if that application is running on GNU/Linux, BSD, MacOS or Windows [note]Stability and quality of the OS can play a big part if it’s too poor, but to Microsoft’s credit, Windows has come a long way in both security and stability since the Windows Vista days. I’d consider all 3 major platforms at a “good enough” stage for most users, where it’s not going to be a factor any more.[/note].

 

We succeeded in the mobile space…

The embedded and mobile computing space played out a bit differently compared to the desktop sector. The traditional proprietary mobile platforms were extremely expensive both in licensing and in regards to development tools. With the lack of a simple major platform (like Windows on desktop) the application market was fragmented, so consumers had no specific reason to choose one OS over another. Free Software wasn’t really a player in any measurable sense, but most of the proprietary platforms were in an equally bad situation.

The mobile application market didn’t really start developing properly until Apple released iOS [note]Of course it’s important to note that whilst Apple’s iOS uses a lot of Free Software in it’s core, it’s a fundamentally un-free platform – just less restrictive and more accessible than many of the proprietary platforms before it.[/note] and Google released Android – suddenly there were two massive players with platforms that anyone could easily join and develop for.

iOS took a premium slice of the market and Android is busy mopping up all the other potential customers. And because the app ecosystem keeps growing it’s harder and harder to compete with it, there’s barely any room left for a third player like Microsoft, even less for the others like Blackberry to get a stake in the market.

It’s not without a slice of irony that Microsoft (whom dominated the desktop space for so long due to their app ecosystem) is now being thrashed in the mobile space for the exact same reason by it’s competitors. But this is the way of the technology world, consumers don’t really care about the OS, they care about the device, the apps and the price. And Microsoft has found itself late in the game without enough apps – and the good hardware and price isn’t enough.

 

 

… but we’re losing Freedoms there still

Of the three major mobile players – Apple, Google and a distant third, Microsoft, only Google’s Android operating system is considered Free Software. And whilst Android has the vast majority of market share currently, if either Apple or Microsoft make gains it will probably come at the cost of Free platform market share.

“Free” is also an interesting world to describe Android. Many would consider it a poster child for Free Software success, but actually it’s more a text book example of how companies are using Free Software to get a head start in the market and then once they have obtained market share, start to introduce lock-in.

Sure there’s Free Software in the core, but all the value added bits are being built as proprietary components, tying users and apps to the Google ecosystem. It’s more like MacOS, sure the core is free, but you need all the proprietary bits to make it actually useful.

It’s somewhat telling in that the only Android distributions are either blessed by Google and carry Google’s proprietary software additions (Play Framework, Play store, Google Search, etc) or tend to be grass-roots hacker based projects.

The only really notable non-Google Android fork is Amazon’s FireOS which runs standard Android apps, but has none of the proprietary Google ecosystem. Of course it’s hardly an altruistic project from Amazon – simply they want to profit from consumers themselves rather than having to give Google a slice of the pie.

What FireOS does show however is how hard it is to really innovate and customise Android – you can do whatever you want until you do anything that Google doesn’t want, like not shipping their proprietary apps or completing with them on products like search. As soon as that happens, you’re locked out of the application store ecosystem and it is very, very hard to make up that market share.

Amazon is a huge player and developers can pretty much just port over their apps without any real work, but they’re still lagging behind the growth of Google’s app store. A free-er Android fork is a nice idea, but based on Amazon’s going, the lack of applications makes it a lot harder than one might expect – the simple barrier of not having access to the app store becomes a massive competitive disadvantage.

Over time it’s likely that more and more of Android will be replaced by various proprietary components. Google doesn’t need to have Android as a Free platform any more now that they’ve obtained the bulk of the market share – they can tell their device manufacturers how they want Android to be packaged to best suit Google’s business objectives. The previously Free Software apps that shipped with the OS can be left to stagnate and in their place, better but proprietary Google versions can be shipped.

And this is just the Android OS and how it’s been treated by it’s parent. Device manufacturers are even worse with companies shipping devices with binary drivers[note]It’s argued that this is actually a violation of the Linux kernel’s GPLv2 license, but it’s a common daily practice.[/note] and firmware update DRM that prevent users from loading on their own builds of Android, resulting in devices that are outdated with no hope of updating only 6 months after leaving the shop.

This was never the dream for anyone wanting to run a Free Software mobile device. And whilst Android has left us better than where we would be in an iOS/Windows Mobile/Blackberry/Symbian dominated world, there’s a long way to go.

 

 

Freedom funds Fragmentation

If an onslaught from companies wanting to create proprietary walled gardens wasn’t enough, we have our own community to content with. Unlike a company, there isn’t a single overarching chain of command in the Free Software community that dictates the path that software will evolve along, so developments take place organically based on specific preferences.

This has worked very well in many cases – one of the upsides of not having a single forced vision is that if that vision turns out to be wrong, the whole movement isn’t damaged. But it also causes fragmentation, seen very visibly with the state of the various Free Software desktops such as KDE, GNOME, XFCE and countless GNU/Linux distributions essentially trying to do the same thing rather than pooling resource to create a single robust platform.

A company like Microsoft or Apple can dictate a particular vision and see the whole company move towards it. They don’t tend to make good vs bad decisions any better than communities, but the lack of fragmentation is often more important that making the technically perfect decision since at least everyone is focused on the one direction.

The rise of MacOS, especially with Free or Open source developers is a particularly interesting example. On a technical level, GNU/Linux is a better operating system – vastly better driver support, not tied to a particular hardware vendor, huge selection of applications easily available via the package managers (think app store on steroids) and arguably more stable at a kernel level.

Yet despite this, more and more developers and engineers are now using MacOS. The fact that the community has adapted technologies from Free operating systems and brought them to the MacOS platform, such as with Brew and Cask rather than working to improve the Free Software platforms with the stability and desktop maturity they were missing is a bit of a damning statement.

I believe the reason for this is that Apple has a strong vision on what the desktop OS should be like and whilst it’s far from perfect, it delivers a single consistent high quality standard. You don’t need to worry about that weird bug that only occurs on Ubuntu 14.04 under GNOME nor do you have to figure out as a vendor how to package an app for several different competing package management systems. And you don’t spend months arguing over init systems.

In short, it makes life simple for people. And when Freedom isn’t a factor in your decisions, then MacOS provides the stability and simplicity that GNU/Linux doesn’t whilst still providing a platform suitable for running the tools loved by engineers such as the various GNU utilities. [note]It’s worth noting that Windows could have been this platform, but Microsoft basically stopped innovating – MacOS isn’t really any more Free than Windows, but it is better engineered thanks to it’s BSD roots. Microsoft could fix this if they tried and with the new direction under Satya Nadella, who knows what could happen.[/note].

Attending a conference of IT engineers, even something pro-Free like the excellent linux.conf.au, there is a visible sea of Apple devices running MacOS. [note]Whilst there are a few whom run Free Software OSes on their Apple hardware, the majority of these Apple users tend to be running MacOS – because it meets their application needs.[/note] This is a really clear failing of the Free Software desktop/laptop, if we can’t even get our own supportive community excited and dedicated to running a Free platform, how do we expect to encourage anyone else?

Given this fragmentation and lack of singular vision, I can’t see Free Software making any  significant leaps in adoption in the traditional desktop/laptop market [note]Which is suffering enough as it is with the onslaught of mobile devices and the fact something that fits in your hand has more than enough capabilities for a user who just wants to browse the web and send some emails. Why buy a traditional computer in this world?[/note].

There’s potentially space for one really strong GNU/Linux distribution to stake a place in the power-user/engineer’s toolbox as desktop of choice, but we’d have to get over our fragmentation problem first and work together to get a strong enough common platform and that seems unlikely. And I can’t see it unseating Microsoft Windows or MacOS as the non-power user’s platform of choice. It’d be more likely to see Android or ChromeOS take a bigger slice of the desktop market than traditional GNU/Linux distributions as mobile/desktop computing converges more.

 

 

But Free Software on the server is still strong right?

Sure. For now.

I foresee GNU/Linux slotting into much more of a commodity/utility space (some might say it already has) and it fulfilling the role of cloud glue – the thing that allows your hypervisor to actually run your application. Whether it’s to run some containers, or as the backbone of a server-less compute platform[note]We *really* need a better name for this. It’s cloud all over again…[/note] the operating system is going to take a much more background role in the coming decade of compute.

Most of us already treat our hardware as an appliance and don’t pay too much attention to it. Do you even still remember how much RAM your cellphone has? 10 years ago it would be unthinkable for any self-respecting geek to not be able to recite the spec of every component in their computer, but we’ve reached a point where it doesn’t matter any more.1 6GB of RAM? 32GB of RAM? Either way it runs your browser, Twitter client and chat client perfectly well. It just doesn’t matter.

The same is becoming true for operating systems and distributions. It doesn’t matter if you’re running Ubuntu Linux or CentOS Linux because either way your Java/Rails/Node/HipsterLang app is going to run on it. Either way your configuration manager is going to be able to configure it. Either way it runs on your favourite cloud provider. Either way there’s a licensing cost of $0. It’s a commodity, like water[note]Yeah OK this analogy fails if you’re living in one of the worse places of the world where fresh water isn’t a free human right, but if you care about the ethics of software, you probably have it pretty good.[/note].

The hayday of the GNU/Linux distribution has come and gone. Other than specific requirements about life frames of platforms and version variations of packages, the mainstream distributions are all pretty much interchangeable.

Even support and patch lifecycle, the golden reasons for commercial GNU/Linux distributions is becoming less and less appealing every year. Traditional support isn’t so useful when the cloud vendor provides excellent support and recommendations around running the platform.

I’ve raised AWS cases before reporting kernel/VM issues and had their enterprise support go as far as to send me links about bugs in the version of the Linux kernel shipped by Ubuntu and then sending me an actual patched version ready to install. And of course I could choose to use Amazon Linux and get support of the full stack from hypervisor to application if I wanted to reduce the risk of having a third party vendor’s software.

Any competent DevOps team can debug and resolve most user space issues themselves, which really tends to be something you need to do to get any kind of vendor support in the first place otherwise it’s a pretty painful exercise. So what’s the point of a GNU/Linux vendor in this new world? Seems redundant when the support is available from another layer.

 

But this is still good news for Free Platforms right? Even if we care less about them and selection reduces, Freedom still prevails?

Maybe not.

My concern is that once the OS becomes a commodity, PaaS is the next most logical step. If you don’t care about the OS and just want your app to run, why not use a PaaS product? Why not consider server-less computing? You clearly don’t run the OS for the sake of the OS[note]See I’m the kind of geek who would run a GNU/Linux, or BSD or Minix box purely because it’s beautiful engineering and infrastructure itself is beautiful, but we’re a rare breed of crazy motherfuckers and we need to remember the rest of the world (whilst crazy) isn’t this crazy. Users want apps, not OSes. [/note].

PaaS platforms are seriously worth considering for new applications and make complete business sense, especially if you aren’t invested in large scale traditional IT infrastructure operations. No sysadmin team? Mix in a few DevOps focused engineers with your full-stack[note]Full stack is a myth and in reality, specialization has it’s purpose, but I do favour teams that are multi-skilled rather than single focus. “My specialty is Java, but I’m not bad with Node.JS and a Linux box” is a much better hire than “I don’t need to care about anything but Java”. “Full Stack” is probably a better (even if incorrect) term than “person who does lots of computer stuff and is useful”.[/note] development teams and hit the ground running with PaaS.

But PaaS is a very good platform for lock-in.

It’s not always – a PaaS that takes a Java JAR file and runs it is (by it’s very nature) going to be quite portable and has minimal lock-in as it’s just a wrapper around the JVM offering a deployer and some management tools.

As customers this sort of PaaS is useful, but we’ve kind of solved the deployment and management problem already with configuration management, containers, cloud APIs etc. We demand more from our providers  – what will you do to make my life as a developer easier? How can I integrate with new products/platforms faster?

We call out, beg for, even demand additional API integration and functionality to make our lives easier. And those features tend to be proprietary and vendor-specific by their very nature as by nature they expose hooks into vendor specific platforms and tools. Any application built on such a PaaS becomes more expensive and difficult to port between providers, let alone to a self-hosted scene.

Once you’re running on a PaaS like this, suddenly it doesn’t matter what OS is running underneath. Whilst the GNU/Linux OS might benefit by some patches upstream from vendors building PaaS products on the platform, changes that provide a specific competitive advantage to your provider might not necessarily make their way back upstream. Many Free Software licenses like GPLv2 don’t require the source code be shared for a hosted service[note]This is why the GPLv3 and AGPL exist, to plug gaps in the modern way applications are run, but lots of key software is on more permissive licenses.[/note].

Your app could even end up on a completely proprietary platform, maybe some kind of Unikernel built for a single purpose and which could be entirely proprietary and custom to a single provider. How long till we someone defining a language unique to their platform? Apple defined their own language recently, a custom language for a specific cloud provider isn’t infeasible as more application workloads move into PaaS systems.

Microsoft can’t make Windows Server as good as GNU/Linux for all-round purposes, but they can easily make it a formidable force in the PaaS world where you don’t care about wanting the bash shell, or needing SSH or the various GNU utilities that make server administration on GNU/Linux so enjoyable. If Azure runs an app for less money and just as fast as Amazon, do you care what platform is underneath?

I don’t believe the GNU/Linux server is going to disappear overnight, or even in the next decade. What I do believe, is that we’ll see a change in the way we consider server-side programming and an increasing adoption of proprietary PaaS platforms built into the major public cloud platforms. And the more we adopt these PaaS platforms, the greater the cost to Free Software in the form of reduced innovation going back into our Free Platforms and the lock-in incurred by users of these PaaS platforms.

 

Any positives or is the future simply bleak?

I don’t know if it’s a positive or negative, but it’s worth noting that the proprietary companies of 2015 are a somewhat different beast from the companies of the ’90s.

In our current (dystopian?) reality, services are available by the second, minute, hour or months and common functionality is a race-to-the-bottom commodity[note]If you’re a VPS/cloud provider and not one of the top 3, I’d be worried right now. You can’t complete on pricing with their scale.[/note]. There’s usually no massive licensing costs to start using a platform, the modern software companies just need a credit card added to an account, or a small fee (like the Apple $99 developer fee) to get started with their technologies.

This is a positive in that at least these proprietary technologies and platforms are more fairly accessible for anyone and not restricted as much as in the past. But this also makes the products and platforms of these companies extra dangerous because they’re nowhere as unreasonable as the proprietary villains of old. And whilst there are plenty of members of the Free Software community attracted to Free Software because of the Freedom aspect, we cannot afford to ignore the fact there’s a huge amount of user adoption and developer contribution that occurred simply because it was zero-cost and easily accessible.

If Microsoft Windows Server had been a $5 download with no stupid incomprehensible license model and the ability to download via a click of a button, would we have had such a strong GNU/Linux adoption today? Or would a lot more innovation have occurred on Windows instead? [note] It’s a model that Microsoft seems to be playing with their mobile device operating system recently. They made the decision to make Windows zero-cost for small devices putting it on equal footing with Android, which they hope will pay off by making it’s adoption more attractive by device manufacturers, and in return application developers.[/note]

I fear that the extremely low barrier of entry of current proprietary technologies is driving us towards a less Free world. And what’s scary this time is that the proprietary offerings look good in terms of polish, price and accessibility.

An over-priced product, or restrictive product makes it easy to promote the merits of Free Software as an alternative, but a proprietary solution that’s cheap-enough, equally good/better technically and just as accessible as Free Software is a much harder sell.

And whilst I believe Freedom should be enough to drive choice on it’s own merits, it’s a hard sell. Companies are worried about building solutions as fast as possible and saving costs, not Freedom [note]Yes there’s a benefit in Freedom to companies in that it offers no lock-in, but companies are exceptionally poor at analysing the cost of lock-in and it’s very hard to allocate numbers against it in a business case.[/note]. Most users are worried about whether they can Facebook on their phone, not if their browser is Free as in Freedom. Hell, it’s hard enough to get people to care if a boat load of refugees sinks off the coast next to them, let alone the specifics of morality and freedom of software. We suck at understanding the consequences of anything that doesn’t impact us directly in the next 5 minutes.

 

Can we fix this?

I don’t see an easy solution to addressing the issues raised in this post. To some degree I believe we’re going to have to accept an increased amount of proprietary technologies in our computing environments and the reality of the majority of the world’s computing capacity being in the hands of a few select public cloud providers seems unavoidable given the economies of scale.

What we can do is try to ensure that we choose Free Software solutions as much as possible, or at least solutions that contribute back into Free Software.

And when we have to use proprietary solutions, avoid ones that encourage lock-in by using generic interchangable solutions rather than vendor specific. You may use a proprietary cloud provider[note]An argument could be made that a non-free cloud provider is akin to using non-free hardware and we’ve replaced one evil for another, but open hardware is a whole another rant for another day.[/note] but run a Free operating system on top of that, knowing that you can always select another virtual machine provider – the virtual machine layer being a commodity.

I strongly believe that for a truly free society, we need Free Software to allow us to prevent being controlled or restricted by any one particular company, organisation or government.

And whilst a lot of people don’t always buy this argument for Free Software, you can’t deny the history of our race isn’t particularly good at respecting individual freedoms even in traditionally “free” societies. Whilst politics tend to change at a pace a bit slower than technology, change they do – and today’s bastion of freedom isn’t always going to be the same in 40 years time.

I’m not going to say don’t use any proprietary solutions – to compete in the current world you need to accept you’re going to have to use them at some level. The current technology scene basically requires you to use a public cloud provider to stay competitive in the market and use various proprietary technologies. But make careful choices and don’t lock yourself in needlessly. And contribute to Free Software where you can, even if it’s just that small internal app you wrote that nobody would ever find useful.

If you’re a Free Software supporter and proponent – awesome, please keep doing so, the world needs more of you. But if you’re lacking direction I’d suggest focusing your energies around privacy and ensuring we can have private computing no matter the platform behind it.

We are in a world where our friends and families will use proprietary platforms no matter what we say or offer as an alternative. As computer geeks we tend to shy away from politics and “patch” anything we think is dumb with technology. Data retention laws? “Meh, my communications are fully encrypted”. Remote snooping? “Doesn’t matter, Free software means no backdoors[note]Well in theory, some members of the infosec community will disagree…[/note]”. We are smart, smarter than the people whom write laws, smarter than the average user.

But we’re not so smart that we’ve been able to make a platform that is so much better than our proprietary competitors and bring Freedom to the majority of computer users. We’re not so smart that we’ve figured out how to make GPG email work smoothly for everyone. We’re not so smart that we were able to prevent the NSA spying on millions of Americans.

We need to realise that ignoring what’s happening in the real world has real impact for those not able to use our free platforms (for whatever reason) and we need to focus some attention on helping and protecting society as a whole, not just our own technology community.

A purely Free software world is perfect. But a world where we have a mix of Free and proprietary solutions is a lot more palatable if we know companies and governments have very tight laws and technological restrictions around what they can and cannot do.

I’d like to know for certain that my friends iPhone isn’t recording what we’re saying. I’d like to make sure someone in a religiously charged restricted country can safely express themselves to their friends without fear of secret police. I’d like to know that governments can’t spy on political adversaries. I’d like to know that the police aren’t scanning people’s private data to dig up dirt and misdemeanors to prosecute them with. I wish these were theatrical examples, but they happen all the time – and those are just the ones we know about.

And I’d like to know that these freedoms are secure not simply because the laws are written to say “don’t do it”, but rather we make it impossible for proprietary software companies to even have that capability to give to a corrupt or malicious government using technology such as client-side encryption[note]Interesting Apple has been showing some positive moves in this space recently, although they have a long way to go.[/note] and OTR.

I don’t believe it’s possible to make a fully Free Software world. I wish I was. And I would love that so much. But based on the current trends we are going to enter a world where proprietary technologies and massive cloud companies play a much bigger part whether we like it or not. And we should work to make that world as best as we can for all involved.