While recovering from OpenStack Summit in Hong Kong, I watched the buzz coming out of the recent AWS re:Invent conference and quietly thanked the scheduling gods at @RedHatCloud for not making me go to back-to-back mega-cloud events.
The OpenStack Design Summit in Hong Kong had roughly 3000 attendees, 2/3 of which were first-time attendees. That’s a good sign that OpenStack is expanding beyond the core contributors & backers and is rapidly gaining traction around the globe.
The last count I heard was somewhere between 8,000 and 9,000 people attended AWS re:Invent. The event grew from last year by 2,500 - 3,000 people. Amazon made several major new product announcements ranging from AppStream, WorkSpaces for Desktop Computing,AWS CloudTrail, Amazon RDS for PostgreSQL, Kinesis, to a new generation EC2 Instances for Compute-Intensive Workloads.
“More” is Awesome if you are Amazon
It was an interesting juxtaposition as the press and clouderati praised Amazon repeatedly for its ever-expanding feature set while pointing to OpenStack’s inability to concentrate its energies on a core feature set and harden itself against expanding its project portfolio.
Not once did I hear any complaining that Amazon had over-stepped it mandate. Instead, there was much kudos & admiration for the way Amazon managed to eat their eco-system partners’ share of the cloud cake and move into entirely new market spaces at such a rapid pace. Amazon’s ambition and their rate of innovation beyond their core market is not seen as something that will sink them, instead it is lauded as one of the key reason why Amazon will be difficult to compete with in the future.
As Amazon is not an Open Source project, we have no real insights into what internal projects failed, were culled, or are on the road map other than what we can cull from the keynotes and AWS-sanctioned presentations. The combination of Bezos’ carefully cultivated corporate culture & their agile develop & deploy philosophy that appears to be along the lines of release lots of feature sets early, often, in a few zones but not all, gather customer feedback, cull carefully and iterate often. And it’s working great for them and for those of us who reap the benefits for their widening portfolio of services. At Red Hat’s OpenShift, we get to reap the benefits of efficiency gained and pass them back onto our PaaS customers in terms of lower prices and can now make our services available in 14 new countries. Amazon’s growth is definitely a win-win for their cloud eco-system.
“The Sky is Falling” if you are OpenStack
Now, compare that with the ‘advice’ given to OpenStackers by analysts, press and assorted cloud vendors leading up to & during the Hong Kong Summit. The OpenStack community was said to be behaving badly by expanding “at an unprecedented pace”, that OpenStack upstream initiatives were “swallowing any potential proprietary value add anyone could add around the core” by adhering strictly to an “open core model” according to Mirantis’ Boris Renski .
That post started off a flurry of dire predictions about OpenStack’s future. Many of them warning the OpenStack community that it had better NOT expand too far into the pockets of the eco-system of vendors that have chosen to participate in the OpenStack community for fear of ‘eating their own supporters’.
Other pundits took direct aim at new OpenStack-related initiatives being floated for the first time at the Summit as ‘knee-jerk" or "half-formed" showing a complete lack of understanding of the open source development process in which openly floating new ideas and concepts is part of the process. It’s understandable that multiple vendors coalescing around flushing the use cases at the conceptual design stage rather than after a chunk of code has been loft into a github repo might seem odd to some, it appears to be a new emerging best practice for open source which is directly arising from the open collaboration being fostered within the OpenStack community.
Spreading FUD boosts readership, and working in the open gives pundits something tantalizing to write about especially when they aren’t given access to what goes on behind closed doors at Amazon.
The whole Core debate is a red herring meant to protect some of the weaker business models that have developed around the OpenStack project and that have not chosen a defensible position in the market. This is not to say that a stabile and mature core is should not be the primary goal for the OpenStack community. What I am saying is that the community has the processes in place and the ability to grow and sustain new projects and initiatives successfully and we should not shirk away from doing so.
Core alone is not enough.
If we listen to the fear-mongers and believe the OpenStack processes cannot support growth beyond core IaaS feature set, and we fail to grow the OpenStack’s own portfolio of features, we risk quickly becoming irrelevant as Amazon continues its proprietary quest for cloud market domination & saturation.
In order to have a competitive Open Source offering for building clouds, both public & private - we need to add new services and features to the OpenStack portfolio as well as mature and stabilize the “core” projects.
A good example of a ‘newish’ feature is the Ceilometer project that aims to become the infrastructure to collect measurements within OpenStack. Its primary targets are monitoring and metering, but the framework should be easily expandable to collect for other needs. To that effect, Ceilometer should be able to share collected data with a variety of consumers. This project’s existence covers a very real use case, and I am sure, has uprooted a few community members initial business plans - but a cloud without a framework for monitoring and metering would be deemed useless by most enterprises.
Continued innovation is critical. Amazon recognizes this, the OpenStack community needs to embrace this ethos as well.
Cultivate our own Garden
The difference between these two cloud giants is that everything OpenStack does, it does in the open. All our successes and failures are in the open.
The OpenStack community is an awesome software factory which has an awe-inspiring process for managing releases with a continuous integration, source code management, peer review tools so much so that one of its community members has packaged up the process itself as a product offering.
Each new initiative gets announced in mailing lists, makes rounds at un-conference & lightning talks, gathers steam, builds a wiki page or two, finds support, gains a github repo (or two), develops a blueprint and if it manages to find a little sunlight, fertile soil - it might just make it into incubation and eventually (if not quite as rapidly as Amazon’s process) be accepted as an official OpenStack project.
There is an open design & development process, and it works.
It can and does support integrating new projects, the process for vetting new ideas and growing a stronger portfolio of new services. The sunlight of transparency only makes the design & vetting process stronger.
Sometimes, OpenStack will cannibalize a market, but more often than not the vendors in that space adapt and help grow the project - because that’s how open source collaboration works. If your IP isn’t significant or unique enough to fend off a little community competition, it’s a good sign that you need to look elsewhere for your value-proposition.
If you are spending cycles worrying that OpenStack is eating your market space, you ought to be watching Amazon.
The companies & individuals backing OpenStack for the most part seem to understand the OpenStack process - even if the pundits don’t. Most OpenStackers tend to take a longer term view of the market space and see the bigger picture.
OpenStack’s focus should be on ensuring that its processes and culture remain open to innovation and are not reigned in by fear of overstepping our ‘mandate’ - our mandate is open and will change as the technology evolves.
As much as we love Amazon and stand in awe of Netflix’s agility to take advantage of new technology and techniques, we’ve all been burned way too many times by software monopolies to ever go back to a world dominated by a single Cloud and that certainly would not be a world that helps others cultivate their own gardens.