Sleeping on the Shoulders of Giants
Why no one is innovating, and what we can do about it.
[n.b. I wrote this article a few years ago, with the context of trying to help people bring a more innovative approach to their organisations.]
There’s a crisis in the world of innovation. Most people cannot explain what innovation is, let alone how to go about it. Even worse than that, according to Eric Weinstein and Peter Thiel, there is very little innovation in the world at all. The oft-quoted “ever increasing pace of change” is largely an illusion caused by one or two industries, and most other industries have been stagnant for decades.
What is so fascinating about this trend is that we know how major discoveries occur. Plenty of invention has happened throughout history, and yet the key principles are ignored by virtually everyone today.
By the end of this article you will understand what powered the innovations of the past and how you can bring those timeless principles to your own organisation.
The Dream Machine
It’s 1969, and a few experienced recruits are taking their first steps into a brand-new building in the hills near San Francisco. The rooms are completely empty; the harsh concrete walls bare.
Included in this list of researchers are legends-to-be like Alan Kay and Gary Starkweather. They, along with a few tens of other staff, were brought here by Bob Taylor, with funding from the US Government Advanced Research Projects Agency (ARPA) and Xerox to the tune of some $12 Million per year.
Fast forward to the present day, and what was invented by this small group has contributed an estimated $30 Trillion to the world economy.
What made this tiny enclave in Palo Alto so incredibly productive? And what could possibly have driven the management of Xerox to destroy it, after many successful years?
The answer lies in the combination of extreme talent, complete freedom and a compelling vision — a formula that has repelled many managers since the days of Frederick Taylor and his scientific management. To quote Alan Kay:
‘They hated [Taylor] for the very reason that most companies hate people who are doing something different, because it makes middle and upper management extremely uncomfortable. The last thing they want to do is make trillions; they want to make a few millions in a comfortable way.’
PARC was set up to take in the most extremely talented individuals, point them in the right direction and give them free reign. When it was founded, PARC’s management effectively said “We want you to invent the office of the future. Go!” As counterintuitive as this may be to most managers, there are several reasons this seemingly out-of-control culture is so wildly productive:
1) Hiring extremely talented people is vastly different to hiring ‘good’ people, because the distribution of individual competence follows an 80/20 rule. Twenty ‘good’ people cannot match one great hire.
2) Extremely talented people are generally highly self-motivated to solve complex challenges, and also tend to find bureaucratic processes suffocating. These are people who ‘have to do, paid or not’ and behave more like artists than employees.
3) A powerful vision for the future can act ‘like a magnetic field from the future that aligns all the little iron particle artists to point to “North” without having to see it.’ (Kay)
During its years of existence, the researchers at PARC invented virtually everything we experience in computing today — from helping to build the early internet (ARPANET) to designs that paved the way for the modern PC (Xerox Alto), laptops and the iPad (DynaBook). As it happens, Steve Jobs and Bill Gates were actually given demos of the Alto, which heavily inspired their designs at Apple and Microsoft.
Perhaps the most important aspect of the ARPA & PARC was the nature of their thinking:
Of the many things not understood about the ARPA-PARC process, the hardest to explain are the levels and kinds of cooperation combined with the willingness and ability to think about “systems” rather than “technologies”. It was the great ability of Licklider [the visionary who was key to setting up ARPA] not to try to “be in control” (and realizing that as things scale you can’t be in control, so other ways have to be devised to make systems that behave closely enough to what is desired).
Alan Kay
Systems thinking was fundamental not only to the management practices of PARC, but to the entire design of the internet itself: ‘The internet architecture they built, based on decentralisation and distributed control, has scaled up over ten orders of magnitude (1010) without ever breaking and without ever being taken down for maintenance since 1969’ (Cummings).
Their work really was decades ahead of its time, and many of their visions of interactive computing and ‘human-computer symbiosis’ have still not come to fruition.
Digging for Gold
Whether it is Xerox PARC, Valve Software, or Bell Labs — why is the formula for extreme productivity so often the same? Why is it that freedom and experimentation are such vital ingredients of invention?
Is there some property of nature that translates them into a recipe for success?
There is — and it is called convexity.
To understand it, we first need to recognise that innovation is always uncertain. You will never know quite what the destination looks like, and your theories and hypotheses must be tested — with experiments.
As Yogi Berra said, ‘In theory there is no difference between theory and practice — in practice there is.’
Now let us create a hypothetical experiment, with some probability of a gain, and some probability of a loss. In a linear case, the chance of making a loss is equal to that of making a gain. If this were a bet at a roulette table, it would look like a bad bet.
Let’s change the experiment. Let us imagine a very risky invention, where the cost of making a mistake is high and the benefit of the invention is low. An example might be Ford’s concepts of a nuclear car. Clearly, the downside (risk of radiation poisoning) outweighs the upside (fewer visits to the gas station). We call this the concave case.
Finally, we imagine the convex case. This is a scenario where there is low risk, and if we do win, we win big.
If you keep making bets in each one of these scenarios, clearly there is only one case in which you come out ahead — the convex case.
Imagine you are part of inventing something world-changing like the internet. Every experiment has a low cost — some circuit boards or some code — but the potential upside is effectively infinite.
If you can just afford to do enough experiments… you might just strike gold. Nassim Nicholas Taleb calls these ‘positive Black Swan’ events — highly unexpected, highly impactful positive events. The difficulty for most funders is that the failure rate at the edge-of-the-art will always be very high. Most managers will consider a failure rate of 75% to be “inefficient,” even though those that do succeed more than compensate for the failures.
The insight of convexity can guide our approach to innovation and provides an extremely good reflection of what successful innovators and entrepreneurs have done in the past.
Limit the Downside
The most important note from this discussion of convexity versus concavity is that it can go both ways. You may not know whether you are making convex or concave bets. As such, you might run in to positive or negative Black Swans.
As Warren Buffet would say, ‘first you have to survive.’ The most important priority is to avoid going out of business, and this can be done by reducing the size of each experiment you do.
You also have to make sure that each of your small losses — failed experiments — do not bankrupt you. Imagine you are a venture capitalist — you may invest in one hundred companies and make a loss on ninety-nine. Your hope is that the one that wins big makes enough money to pay for the small losses.
Keep Your Options Open
As we explored above, ‘You can’t innovate without experimenting’ (Bezos). The consequence of convexity is something called the ‘1/n’ rule. That is, if you have a budget, you should maximise the number of individual experiments, splitting the budget as finely as possible. You then rapidly iterate from experiment to experiment, learning as you go. This is called a “blind search” technique.
The key here is that you must be free to change your plans from what you learn in each iteration.
‘A rigid business plan gets one locked into a pre-set invariant policy, like a highway without exits — hence devoid of optionality. […] To translate into practical terms, plans need to 1) stay flexible with frequent ways out, and, counter to intuition 2) be very short term, in order to properly capture the long term.
‘[…] This explains why matters such as strategic planning have never born fruit in empirical reality: planning has a side effect to restrict optionality. It also explains why top-down centralized decisions tend to fail.’
Nassim Nicholas Taleb [here]
Researchers are a kind of Artist
As explored above with Xerox PARC, extremely talented people are usually motivated to solve complex challenges on their own. By providing them with a clear and aspirational vision to work towards they will all pull in the same direction with astonishing results.
In the case of PARC, this was J.C.R. Licklider’s vision of “human-computer symbiosis.” For Apollo, it was putting man on the moon. This purpose was so pervasive that when President John F. Kennedy asked a janitor what he did at NASA, the janitor famously replied: “I’m helping put a man on the moon!”
But this is not the same as providing people or teams with goals or targets. Goals limit the scope of the imagination and are inflexible to new learning and ideas. Likewise, they make it difficult to justify funding opposing approaches to a problem — a fundamental requirement if one is to support experimentation.
As Kay explains, what creates exceptional results is funding extremely talented and artistic people, giving them freedom and a compelling vision:
‘[I]t is no exaggeration to say that ARPA/PARC had “visions rather than goals” and “funded people, not projects”. The vision was “interactive computing as a complementary intellectual partner for people pervasively networked world-wide”. By not trying to derive specific goals from this at the funding side, ARPA/PARC was able to fund rather different and sometimes opposing points of view.
‘The pursuit of Art always sets off plans and goals, but plans and goals don’t always give rise to Art. If “visions not goals” opens the heavens, it is important to find artistic people to conceive the projects.
‘Thus the “people not projects” principle was the other cornerstone of ARPA/PARC’s success. Because of the normal distribution of talents and drive in the world, a depressingly large percentage of organizational processes have been designed to deal with people of moderate ability, motivation, and trust. We can easily see this in most walks of life today, but also astoundingly in corporate, university, and government research. ARPA/PARC had two main thresholds: self-motivation and ability. They cultivated people who “had to do, paid or not” and “whose doings were likely to be highly interesting and important”. Thus conventional oversight was not only not needed, but was not really possible. “Peer review” wasn’t easily done even with actual peers. The situation was “out of control”, yet extremely productive and not at all anarchic.’
Alan Kay
PARC Life
As we saw before, the Xerox management was often shocked by the ‘out of control’ nature of PARC. They didn’t understand that the vision and good will were so compelling that the researchers continued to deliver breakthrough after breakthrough with remarkably little guidance or ‘direction’.
Despite these breakthroughs, two pressures would prove to be the nails in PARC’s coffin.
First is Warren Buffet’s “institutional imperative” — the intense inertia that comes with bureaucracy and the tendency of mediocre managers to eject extraordinary productivity like the immune system ejects a virus.
Second is the bizarre myth that nothing commercial had ever come from PARC. ‘Absolute bull***t’ says Kay; in reality, Xerox made billions on the laser printer alone. Xerox’s ROI was over 250x.
Despite this, Xerox’s management hated PARC and in 1983 Xerox ended up firing Taylor — the man who made it all happen.
This sort of culture has been created elsewhere — see Y Combinator or the Santa Fe Institute — but this sort of free experimental research is becoming increasingly rare and increasingly constrained by rules around R&D funding.
The principle of funding ‘People not Projects’ would be blocked today in the name of preventing of nepotism and corruption. It is telling that the USA’s special forces, JSOC, routinely uses special processes which lay outside the normal law to avoid delays in procuring technologies. The standard legal and procurement processes of most developed nations are so arcane and bureaucratic that they actually empower corruption and cronyism — only large corporates can afford to navigate the rules, often with the paid help of the ex-ministers and civil servants who actually designed the regulatory systems.
Similarly, most of the management practices that produced the major aerospace innovations of the 20th century would now be ruled illegal. George Mueller, whose transformation of NASA made Apollo a success, stated that Apollo would be impossible with the current legal system; it could only be done if it were classified as a ‘black’ programme. Today, when significant developments do occur, they are increasingly created in Skunkworks-style, secretive outfits, hidden from bureaucratic scrutiny.
Even venture capital has gone slightly astray: The famous Bell Labs facility was said to have signs stuck to the walls saying, “Either do something very useful or very beautiful.” According to Kay, most VC funders today would be hard pressed to fund the former, let alone the latter.
[n.b. I have cut out most of the summary section that went here.]
I will leave you with a quote from Alan Kay which is a perfect antidote to today’s inescapable insistence on forecasting and anticipating the future:
“The best way to imagine the future is to build it.”
