What does the shift to multi-cloud-OSS-aaS mean for COSS companies/founders?

This week, Google Cloud announced GKE On-Prem. Despite the fact that I think this is a really suboptimal product name… 😏

GKEOP doesn’t quite roll off the tongue. Can we just call it kompute?
— Joseph Jacks (@asynchio) July 24, 2018

…I do believe this represents a major strategic transition away from cloud providers’ historical stance on singularly optimizing for gravitationally migrating customer/enterprise compute cycles to their data centers... and instead also meeting workloads (mostly legacy, hard-to-move, heavily regulated ones) where they are. Kubernetes is a huge enabler of this:

Over past decade, s/w vendors pursuing “multi-cloud” products assumed that clouds themselves would never be incentivized to standardize interfaces..That has completely changed today. Multi-cloud is now a first-class value prop of the leading clouds. Thanks to @kubernetesio.
— Joseph Jacks (@asynchio) July 24, 2018

At the Google Next event this week where GKE On-Prem was announced among many other new products/services/regions, “multi-cloud” was clearly a major theme. Kubernetes enables a multi-cloud reality as it commoditizes compute primitives over a declarative API and open source runtime that can live anywhere — at the edge, on embedded devices, in data centers, in the cloud… and more.

So, what are the implications of clouds (not just GCP, but also Azure and increasingly even AWS) embracing “multi-cloud” for COSS founders/companies? First, let’s unpack what multi-cloud is really all about:

Defining “Multi-Cloud”

I don’t have to write much here because two people I have a huge amount of respect for just did ... timely!

Here’s Mitchell’s take on the various dimensions (100% agree with him here):

Multi-cloud has multiple orthogonal dimensions. Each has its own different value and own different challenge, but you can be “multi-cloud” with only one: 1) workflow portability 2) workload portability 3) traffic portability 4) data portability. I consider #1 as most important.
— Mitchell Hashimoto (@mitchellh) July 25, 2018

"Why Multi-Cloud?" is a topic I commonly see asked. I saw this Reddit thread and used that as an opportunity to type a fairly detailed (but still brief) answer. https://t.co/GwgxeEzsXl
— Mitchell Hashimoto (@mitchellh) July 24, 2018

And here’s Bassam’s take (open this one/click for thread):

Interesting times in #CloudComputing as we collectively grapple with the transition to an open and #multicloud world. Early days with lots of exploration/confusion. Here are some questions that need to be answered before we start to see convergence #googlenext18@CloudNativeFdn
— bassam (@bassamtabbara) July 25, 2018

OK —these perspectives are great for understanding multi-cloud. Let’s now double-click on why multi-cloud / cloud-based OSS services is a very GOOD thing for COSS founders and projects overall.

Validation

Why is it a very good thing for COSS founders/companies/projects when cloud providers “co-opt” their/a fast growing and exciting new OSS project as a new hosted service? Validation!

Let’s unpack what I mean by validation by looking at a few awesome examples of extremely successful yet relatively new OSS projects that have varying characteristics, origins, ecosystems and implications…

GraphQL

Characteristics: GQL is at the intersection of new statically-typed query languages, API semantics, architectural paradigms and query formats. It represents a huge shift in the way responsive web UIs, server and client-side APIs and services (microliths and monoliths alike) are built for the future. Riding the utterly explosive and insanely momentous wave of React, GQL is taking the front-end and ultimately, IMO, the server/infra-side programming worlds by storm.

Origins: GQL was created at Facebook almost by accident in 2015. Here’s a great video by my friend Nick Schrock, one of the creators, on the origins of GQL and the road ahead:

Ecosystem: GQL’s vendor ecosystem is huge. It is absolutely a multi-vendor ecosystem. Facebook is the largest corporate user and governance entity (although there is talk/rumbling about GQL moving into an OSS foundation, but we should stay tuned on that)…Apollo ($30M+ in funding from a16z and others) is building API gateways and services (shifting away from Meteor.js to being “The GraphQL company”), Prisma ($4.5M+ in funding from Kleiner Perkins and others) are building the universal data layer powered by GraphQL and have become the de facto community builders for GQL. Here's a slide deck detailing the GraphQL ecosystem.

Cloud<>Vendor Implications: AWS released AppSync, a GQL-aaS offering, late last year…

This is huge validation for the following reason: GraphQL is now elevated to the awareness of the entire AWS customer/user base and ecosystem, broader distribution of visibility, credibility, maturity, etc. How is this a bad thing for the GQL project? It isn’t! Signal that AWS offers a first-class service of any OSS project is always a very good sign that the project has hit a nerve, level of impact and seriousness in the industry that AWS takes notice. This is also a good thing for the COSS ecosystem companies I mentioned before (Apollo, Prisma, others) since they are able to better focus on their differentiated head-start and developer mind-share which AWS often has a very hard time competing with, especially since the first several versions of new AWS offerings are light / poor implementations that are generously described as “MVPs” and only serve the bare minimum features that “cloudify” OSS as a service.

Kafka

Characteristics: Kafka has also taken the world by storm as the standard pub/sub, message queue, commit log and eventing database all-in-one! However, I’ve expressed concern over this in the past:

I worry @apachekafka's protocol-level opinionation coupled w/ the convergence of mediation patterns (ala ESB era) will ultimately create very brittle architectures. One nice property (very different system, but similar shifts happening) of @EnvoyProxy is protocol-agnosticism. https://t.co/YzZwhuJGlX
— Joseph Jacks (@asynchio) June 24, 2018

Origins: Kafka was also born inside a large internet company — LinkedIn. Over time, it was donated to the Apache Software Foundation (ASF) where it is now a top-level project.

Ecosystem: Kafka is really a single-vendor controlled ecosystem. Confluent (currently $80M+ in funding by major tier-1 VCs) was founded by the creators of Kafka after they ran it in massive production scale at LinkedIn and left years after. Confluent has been doing a tremendous job of supporting the enterprise use community, unearthing major use cases that span existing categories and industries: service discovery, data persistence, logging, ETL, data migrations and more.

Cloud<>Vendor Implications: Arguably, AWS has Kinesis, a proprietary version of Kafka and an AWS service they released in 2013…

There are rumors that AWS will release a native Kafka service later this year at Re:Invent, but if they do I believe this is a good thing for Confluent and the Kafka ecosystem overall, as well — it signals that Kafka has reached a level of mass adoption and criticality to AWS (via customer demand/signal) that they won’t hesitate to offer an OSS service that competes with a highly sticky and profitable one: Kinesis. Also, I believe that enterprise customers will ultimately mostly gravitate towards working with Confluent vs AWS for product and support (*aaS/Confluent Cloud) consumption vs. an MVP-implementation of Kafka on AWS. While competition is never fun, it does help expand markets, drive further adoption and create larger commercialization opportunities overall. Unlike Peter Thiel, I do not subscribe to the view that competition is the root of all evil. Sometimes it is a good thing.

Kubernetes

Characteristics: K8s is really eating compute… it is software defined compute at its best. A declarative API for distributed systems with a focus on automating all the tedious and complex aspects of right-sizing compute with modern compute workloads of all kinds (batch, real-time, services, data-intensive, ephemeral, cron, etc). K8s is viewed by many as the POSIX for distributed systems. I see it as commoditizing cloud computing as a fundamental layer and beyond — even from the chip level through to edge devices.

Origins: Kubernetes was also, unsurprisingly, built at a major Internet-scale company — Google. Over the first year leading up to the 1.0 release, Kubernetes was donated to a foundation: CNCF, within the Linux Foundation.

Ecosystem: Kubernetes is unquestionably a multi-vendor ecosystem and there is now an absolutely enormous ecosystem landscape around K8s which is representative of most of the projects coming into the CNCF and the pipeline thereof.

Cloud<>Vendor Implications: The implications of Google’s announcement of GKE On-Prem for commercial Kubernetes startups / vendors are potentially scary, i.e. Heptio..However, overall, I think this is very positive — Google is probably best situated in this move to acquire Heptio or translate Dave Rensin’s Customer SRE org to a field-focused enterprise support group for Kubernetes customers who’d love to pay Google to run Kubernetes in their own data centers without needing to shift large complex workloads to GCP.

Embedded inside every GKE onprem server is a Google SRE
— . (@cloud_opinion) July 24, 2018

OSS eats the cloud

Over time, I believe OSS eats cloud computing. The primary reason Google Cloud is embracing OSS is to disrupt themselves, platform services and leap-frog their way to dominance over AWS. This is a long-term play and one that forces faster commoditization in the cloud platform ecosystem, pushing AWS to embrace OSS and come to Google’s turf (OSS-jedi strength turf).

I take this as a major sign that the smartest people in cloud computing see that the real disruptive force for the future of software is not cloud computing, but open source computing. This is a multi-trillion dollar opportunity large enough to enable the spawning of next-gen cloud platforms that will very likely not pour concrete, own data centers or lay undersea cables.. instead, they will be COSS companies.
“The Well” (We Are Coming Out of It!). M.C. Escher. Wood engraving. 1946.
Rapid growth OSS projects unlock “hidden” potential efficiencies that serve to disrupt enormous, major incumbent platforms and industries that have yet to even fully mature (i.e. cloud computing). That’s why I felt it appropriate to use the above Escher lithograph! 😄