As a VC at Icon Ventures and a twenty year veteran of productizing and marketing high tech for VMware, Netscape
and others, I've always been fascinated by how new technologies emerge and come to market. One of the major
artifacts that tries to capture the state of our market and industry each year is the annual Gartner Hype Cycle - which
I always read with interest. Just last month, I had an interesting thought: "Has anyone gone back and done a
retrospective of Gartner Hype Cycles - because I'd totally read that article". A quick Google search didn't turn up
anything useful, so I decided I'd do the work and write it myself. This article is the result.
As most of you know, the Gartner Hype Cycle for Emerging Technologies is practically an institution in high tech. First
published in 1995, the Hype Cycle proposed a standard adoption model for new technologies. In this model,
technologies all go through a process of :
Emergence: "The Technology Trigger"
Excessive enthusiasm: "The Peak of Inflated Expectations"
Excessive disappointment : "The Trough of Disillusionment"
Gradual, practical adoption: "The Slope of Enlightenment" and "The Plateau of Productivity"
By way of illustration, below is the first Hype Cycle - from 1995. And it's truly a fascinating historical document. Some
of its technologies that have become so ubiquitous, that they're now background noise (Object-Oriented
Programming). Some technologies have simply disappeared from public consciousness (Emergent Computing). Still
others are technologies that we thought were almost baked but actually took decades longer to reach full maturity
(Speech Recognition).
The most hyped technology in 1995 was Intelligent Agents. Two years later, Office 97 introduced Clippy, the
enthusiastic, but incompetent assistant which was so poorly received that it effectively killed off the idea for a
generation. Today, twenty years later, we’re once again trying to build intelligent assistants, although now we call
them Chatbots, and the core tech - contextual reasoning in a broad domain - is still a hard problem.
I think of the Gartner Hype Cycle as a Hero's Journey for technologies. And just like the hero's journey, the Hype Cycle
is a compelling narrative structure. When we consider many of the technologies in use today, we tend to recall that
they were overhyped when they first arrived, but eventually found their way to mainstream usage. But ... is that
really how technologies emerge and gain adoption? After analyzing every Gartner Hype Cycle for Emerging
Technology from 2000 to 2016 - all seventeen years of the post dotcom era - I’ve come to believe that the median
technology doesn’t obey the Hype Cycle. We only think it does because when we recollect how technologies emerge,
we're subject to cognitive biases that distort our recollection of the past:
Hindsight bias: we unconsciously "improve" our memory of past predictions.
Survivor bias: it's much easier to remember the technologies that succeed (we're surrounded by them) rather than
the technologies that fail.
1
www.linkedin.com
Página 1|5
For example, Emergent Computation - the earliest technology on the 1995 Hype Cycle up above - is a great illustration
of survivor bias. (Emergent computing, by the way, is computing based on distributed evolutionary algorithms - a
kind of cousin to neural network based machine learning. (I had to look it up)). Today if I asked 20 Silicon Valley
technologists to name which technologies succeeded and failed since 1995, I think I can guarantee that no-one would
name Emergent Computation. But yet, there it is in the 1995 Hype Cycle, important enough to merit one of just ten
slots in that year's listing.
(And incidentally, my intention is not to call out Gartner’s accuracy as a firm specifically. With some notable
exceptions, such as the technology terms that Gartner coins itself, I think of the Gartner Hype Cycle as mostly a
reflection of industry consensus.)
But our inability to remember the past in proper context is not the only lesson from taking a deep dive into Gartner's
past Hype Cycles. After analyzing every year from 2000 on, I think I can say with confidence that we are simply not
very good at predicting the future. I've learned that lesson and seven more from my deep dive into the data. Read
on for the details:
Lesson 4: The technical insight is often correct, but the implementation isn't there
I was often struck by how many times the Hype Cycle had an insight that was essentially correct, but the technology
or the market just wasn’t ready yet. Some of the best examples:
WS-Enabled Business Models: riding on the wave of enthusiasm for WS* based web services in the early 2000's, WS*-
enabled business models first appeared on the Hype Cycle in 2003 as a declining tech, and appeared again in 2005 at
the Trough of Disillusionment. For those unfamiliar, WS* was a set of technology standards that used XML formats
over SOAP as the basis for remote API invocation. The WS* movement essentially collapsed under its own technical
bloat. Developers hated writing to the API's and there were many specification problems. But yet, the core insight
was essentially correct. In the last few years, the rise of Twilio, Plaid, Checkr and others has validated the essential
insight made all the way back in 2003: that there would be successful businesses whose only product offering was a
valuable API.
Public Authentication Services: First appearing in 2002 as an emerging technology, this prediction was presumably
based on the release of Microsoft Passport - a Microsoft public authentication service based on WS* APIs. Passport
also failed because of its WS* foundation. It took the development of Oauth in 2007 by an informally cooperating
group from Google, Twitter and Magnolia to make the promise real. Today, of course, third party public login via
Google, Twitter and Facebook is ubiquitous. Again, an insight that was correct but the implementation was faulty.
Tera-Architecture: listed as an emerging technology in just two Hype Cycles - 2006 and 2007 - I was unfamiliar with
the term and had to look it up. Astoundingly, Tera-Architecture (a term that Gartner coined but then seemingly
abandoned) described something quite close to a modern data ingestion pipeline for big data. It's true that it took
the release of open source projects like Kafka and Samza to make very large scale ingestion pipelines widespread,
but the essential insight into its requirements were all described by Gartner in the mid 2000’s in a solid piece of
prognostication.
Lesson 5: We've been working on a few core technical problems for decades
There are a number of core technologies that appear again and again in different guises over the years in Hype Cycles,
sometimes under multiple aliases. Each reincarnation makes progress and leaves lessons for its successors without
really breaking through. These are the technical marathons of the Hype Cycle.
Speech recognition: as we saw earlier, Speech Recognition appears in the very first Hype Cycle in 1995, where it's
climbing the Plateau of Productivity. In reality, speech recognition was far from mature in 1995. It's only - possibly -
with deep learning breakthroughs in the last two years that we have reached human equivalent recognition in speech
recognition. Two decades later.
Internet micropayments: alternatively known as ecash, epayments, cryptocurrency and in its latest incarnation -
bitcoin - we've been working on enabling small internet payments between untrusted parties since the dawn of the
internet. Today micropayments still only work well in walled gardens like the Apple ecosystem. Perhaps it will take
another generation beyond bitcoin for this to really work.
Data analysis: large scale data and content analysis has come and gone from Hype Cycles in three waves over the last
two decades: data mining (90's) gave way to analytics (2000's) which in turn gave way to big data (2010's). It seems
to always take a new generation of architectures to cope with the inexorable expansion in the scope and size of data
we want to analyze, and who we expect to perform that analysis.
Página 3|5
Context delivery: it's been obvious for many years that capturing and brokering context is crucial to everything from
content to commerce, but a general approach to this seems to continue to recede. This doesn't stop the term from
continuing to pop up on Hype Cycles as an early stage tech.
Lesson 7: Lots of technologies make progress when no-one is looking
Look at enough Hype Cycles and you can see a pattern where many technologies make steady and sometimes
breakthrough progress after they're considered played out. Like machine learning in the aughts, many technologies
are doggedly moved forward by researchers, startups and large tech companies when their previous generation is
widely seen as having failed. A couple of my favorite examples:
Head Mounted Displays: The first generation of head mounted displays appeared in the late 1990’s and made it onto
the Hype Cycle for 2001. But limitations of that era’s screen technology made the first generation dead on arrival.
It's only in the last few years, the arrival of far more advanced head mounted displays for VR and AR has spurred a
renaissance of hope. Enabled by higher resolution and higher frame-rate displays, cheap motion sensors, and a new
generation of machine learning algorithms, these new displays incorporate almost two decades of software and
hardware advances. (And they're still not quite there.)
Speech Generation: speech generation, text to speech and speech to speech translation appear on multiple Hype
Cycles (2002, 2005, 2006...) But again, it took deep learning breakthroughs to finally generate near-human
performance just in the last year.
For technologists looking for out-of-favor technologies that may be biding their time before their next breakthrough,
past Hype Cycles can be a fruitful source of ideas. Some of the most intriguing technologies that in my humble
opinion, may be due for their second or third round of visibility:
Peer-to-peer computing: last seen on the Hype Cycle in 2002, (although you could argue that Ethereum constitutes
the second or perhaps third coming of peer-to-peer) there are plenty of interesting advances in peer to peer research
over the last decade that could enable a new generation to emerge.
Business process engines/platforms: listed in the 2005 Hype Cycle, it was expected that general purpose rules engines
would power the next generation of business apps. Instead, we got highly functionalized applications like Salesforce
and Workday. Perhaps IFTT, Zapier and others herald a renaissance of business process engines?
Lesson 8: Many major technologies flew under the Hype Cycle radar
One final lesson. It's remarkable the number of major technologies from the last 20 years that were either identified
late or simply never appeared on a Hype Cycle. In technology, so many things that look trivial or transitory turn out
to be the foundation of the next generation of business and consumer platforms. A brief list of the technologies that
should clearly have been listed as important emerging technologies since the dot com wave:
x86 Virtualization: arguably the single most important new data center technology of the last decade, pioneered by
VMware
NoSQL: the massive adoption wave of non-SQL based databases that started in the mid-2000's and gave us MongoDB,
Cassandra, Redis, Couch and more
Map/Reduce/Hadoop: the foundation of this generation of large scale data analysis
Open Source: the proliferation of open source as a licensing model that resulted in the commoditization of
infrastructure software, the rise of community code sharing and the enablement of cloud models
Those are just a few of the major technology trends that never surfaced as part of an Emerging Technology Hype
Cycle - although many of them appear on the various functional, vertical and specialist Hype Cycles that Gartner has
produced in ever increasing diversity over the last decade.
The Lessons About The Lessons
What I take away from my analysis of these Hype Cycles is not just how difficult it is to make predictions, and how
much wasted effort goes into technologies that doesn't tend to work, but also how exciting and wondrous is the
progress that we've made in technology. The labor of the last two decades has given rise to an Age of Wonders: of
self-driving cars, computers that almost understand us, and unfathomable scales of data analysis. More than ever, I
feel personally privileged to work among and invest in the teams creating our technological future.
Update: If you liked this post, you might also enjoy my deep dive into the Magic Quadrant - looking at 16 years of
data for the Business Intelligence category.
---------------------------------------------------------------------------------------------------------
Appendix: The Raw Data Dump
For my analysis, I tracked down every Hype Cycle for Emerging Technologies published by Gartner between 1995 and
2016, and created a list of all individual entries by year. I decided to discard 1995 through 1999because the dotcom
technology categories seemed mostly irrelevant to our current technology conversations.
Página 4|5
Next, I merged multiple terms into a single category if I felt that the difference between the two terms was minor.
For example, I decided that "Personal Fuel Cells" and "Micro Fuel Cells" really referred to the same thing. As another
example, I decided that Context Delivery Architectures, Context Brokering and Context Services all centered around
the idea of Context abstraction, so I treated that as a single term. I'm very aware that I may have merged over-
aggressively, but some of the more obscure and older terms are poorly documented on the web, so I'd be happy to
receive feedback on how I could improve here.
Next, in order to track movement of each technology over multiple Hype Cycles, I coded each entry by where the
term appeared. To code position, I came up with an eight point scale for Hype Cycle position, from stage 1 (earliest)
to stage 8 (mature). My general categorization of the stages can be seen in the figure below.
Lastly, I color coded each entry from light red (stage 1), to deep red (stage 3) all the way to green (stage 8). And using
Excel's value based color coding, I interpolated those colors across the range. What this gives is what I think as a
nicely efficient 10,000 foot view of Hype Cycles from 2000 to 2016.
Below is the resulting database of Hype Cycle entries sorted by year of first appearance. For example, the first entry
- Synthetic Characters (analagous to our current chatbots) - first appeared at Stage 2 in the 2000 Hype Cycle,
appeared again in the 2001 Hype Cycle at the same position, and then never appeared thereafter.
Página 5|5