Momentum

At Mozilla and OSAF I worked on ancillary projects, projects that fed in to the development of the main product without being part of the main product itself. The most frustrating part for me was that while both the main product and my ancillary projects are open source they were hugely different in their needs and community. The main project is well funded, there are many people paid to work on it, and this injects continual investment in to that project, while my ancillary project is usually just me and in order to attract contributors I needed to understand how momentum works in community driven projects outside the walls of my employer.

This made me more attentive to other projects, to what drives momentum and what discourages it. How do you grow a project and what are the limitations? When node.js came along I actively tried to steer things in a direction I thought was positive and would encourage growth but only now, after years of reflection, do I begin to understand why it was so successful and why it now appears to be maintaining an absurd growth rate well above its contemporaries.

The earliest months of a project are the most important. After initial release you have nowhere to go but up, if you release something on your own you're just one developer and if you get 10 people to invest in your project that's 10x growth. From what I've seen there is no real limit to the early growth of an open source project, there are an unbelievable amount of people who will choose to invest in even the newest and least proven technology. You don't need to hitch your wagon to an already growing community and you also don't need to attack the competition to get attention. You need to reduce negativity, that means reaching out to people who are likely to think your project is interesting and distancing yourself from any predictable negativity until you've grown large enough you can handle it. Negativity early on is a killer, it decreases early investment and a new project must avoid it.

Negativity comes in two flavors, internal and external. Internal negativity is attacking the competition; their concurrency patterns suck, you can't write real programs, etc. This attracts the wrong kind of people, you want people who are going to sit on IRC and answer questions in their free time not troll every newcomer. In the early days you should hope that anyone you would attack decides to ignore you, completely. Human beings make investments and any project that is incompatible with their current investments is a potential threat. This is where external negativity comes from and the best way to avoid it is distance.

Every platform comes with a set of established patterns for compatibility. Sometimes these patterns are incredibly limiting, some of them may even lead to the eventual demise of the platform itself. Every platform has compatibility forks that attempt to overcome those limitations and in every case I know of the fork fails to overtake the established pattern. Anyone who's written Twisted or EventMachine knows what I'm talking about. These forks get some momentum early on but are eventually crushed by the negativity from the rest of the community they try to make their home.

If you write a library in Twisted you can grab some value from the rest of the Python community (the language, a portion of the libraries) but a lot of it isn't compatible with what Twisted is trying to do and what you publish using Twisted isn't usable by the rest of the Python community at all unless they adopt Twisted, which leads to some predictable negativity from the community at large.

But, the real question should be "why was Twisted trying to be compatible with Python at all?"

Promises to Fibers

At this point in time Promise libraries for node are entirely compatible with all established patterns in node.js and are only incompatible if the author chooses not to expose a compatible API. If you use a Promise library in your module it's at your discretion whether or not you expose an API as a Promise or as a callback which you then internally wrap up in a Promise. In many ways Promises are just another flow control library, in node at least, and when authors choose to expose a callback API they are opting out of being a compatibility fork. This means two things 1) that Promises are highly unlikely to grow their own community that is incompatible with most of the node ecosystem and 2) the usage or Promises as an external API will never eclipse node's currently established patterns.

If the goal of Promise libraries is to overcome some limitation in node's patterns then they are, currently, limiting the scope of that fix to people who actively invest additional resources in supporting Promises. They are not building a significant ecosystem for their pattern because they are hampered by compatibility. This isn't necessarily a bad thing, this could be the goal of Promise libraries, to provide a flow control pattern for people that seek one out and not to create something potentially bigger than node that is free of the limitations they see in node.

Fibers and streamline are another bag of worms. Their advocates are small but loud and live on the node.js mailing list. Both libraries are attempting to overcome limitations they see in node.js and both do it in ways that limit compatibility with the rest of the ecosystem. Both are met with a fair amount of negativity, which is unfortunately predictable, and it has slowed their growth rate to a fraction of the rest of node's. I have the same question for these libraries as I do for Twisted; "why be compatible with node.js at all?"

Myths of growth

Embedded deep within our thoughts on this subject is a false belief. We all seem to be under the impression that the available growth to open source ecosystems is finite. The reason why people view new challenges to their investment as a threat is the idea that if it were to take off the project would reduce existing investment compatible with established patterns.

The reason partially incompatible forks try so hard to be at least partially compatible is so that they can avoid having to create their own community and gain their own momentum because they believe the available investment is finite and that its better to push their way in to an established and growing community than create a new one.

What I've found is that the available investment in open source ecosystems is not finite. Investment in new projects appears to have little to no impact on existing projects. What I do find is that there are limits to momentum. Quarter over quarter percent growth is dramatic in the earliest growth period (there's so much room to grow!) but once that growth, as a percent of the whole, reduces it rarely if ever grows more than 5% above the previous quarter.

Momentum is not a series of curves that dip and swing, momentum is a snowball on a mountain. The snowball experiences dramatic growth as it starts down the mountain but once it begins to flatten its potential is limited to maintaining its speed or slowing down.

Releasing a new project is the top of the mountain, the more speed you can give it in the earliest phases, the longer you can keep it from slowing down, the greater the potential for that project in the long term.

This is why I think partial compatibility is a mistake. Partial compatibility exposes you to more negativity, which limits early growth. Partial compatibility also limits the amount of potential investment. Node.js could have been a non-blocking fork of Narwhal, it would have had the "advantage" of many existing js modules and the potential to bind to the huge world of Java libraries. Node took a different path.

Node is one of the least compatible platforms that has ever been released. It was compatible with almost no existing JavaScript code since nearly all existing code used the DOM (it would be almost a year before jsdom and others created node versions of the DOM). It was incompatible with most C libraries that nearly all other platforms lean on, like database bindings, which meant that new libraries had to be written from scratch where they hadn't been for platforms like Ruby and Python. There was almost nothing written that worked on node and nothing published that could be used to write a node application. The Fibers and Twisteds of the world consider this incompatibility a liability, a huge barrier to investment, when the opposite is actually true.

In the earlier days of node you could write anything. The advantage of nothing being compatible is that there is a much greater opportunity to attract people interested in writing virtually anything. That means a greater potential for early investment, which creates early momentum, which you can only lose over time so maximizing that early growth is the most important thing.

Node made very few decisions, it limited the size of its core much more than other platforms, but decisions were made. One of those decisions will eventually hold it back from being suitable for a future class of applications. I'm confident in this because I've seen it so many times before. Whatever takes off for that new class of applications to eclipse node will not take off inside the walls of node's ecosystem. Attempting to innovate against the grain of node's current patterns for compatibility will insure failure the way other compatibility forks in other languages failed.

If you take anything from the success of node's ecosystem it's that if you want to create something that overcomes node's limitations you can't do it within node's ecosystem.