The ...................................................................................................................................... 9
◦ Error Pages............................................................................................................... 11
◦ Email Receipts.......................................................................................................... 19
◦ Status Pages............................................................................................................. 39
◦ Changelogs............................................................................................................... 50
◦ In-app Communication........................................................................................... 76
Conclusion ............................................................................................................................ 80
Credits ................................................................................................................................... 83
Things like Lean and Agile have come along to help combat waste when it comes to
building really anything. But when it comes to eliminating it, the methods are far
from a silver bullet for SaaS. There is still a lot to study and learn. Where some
products seem to invite success at every turn, the vast majority struggle to just
barely keep their heads above water.
We began to think: “Is there something systemically different about those products
that succeed that could be distilled out and applied to those that struggle?"
Over the past decade, all the people at Ramen have worked on a huge number of
SaaS products in various capacities. But it’s been these last two years in particular
that we’ve begun to really focus on that question of why some products seem to
“have it easier." We started to pay attention to SaaS products in any vertical, and
despite being in different industries, we began to notice some similarities among the
successful products. What first started as loose anecdotes quickly turned into
repeating patterns. We saw again and again that products, no matter how different,
could do similar things to increase their overall quality regardless of how different
the products themselves might be.
Product Success is about what happens at the product periphery, and being
thoughtful about every single interaction with customers in the hopes that it will
increase a customer’s trust in the product and their knowledge about the product
team’s intentions. In turn, this has the effect of raising the level of discourse between
customer and team. These higher quality feedback loops result--very directly--in
better products.
If you don’t want to read the whole book, then here’s a TL;DR for you: First, share the
book with a friend if you haven’t already and then think about this question when
working on a product:
What are the “boilerplate" things that you just do because you’re supposed
to?
The thing about this boilerplate is that they are common across all SaaS products. So
even if you have no competition in your space, your customers will judge you on the
quality of these interactions. Email receipts, About Pages, Availability of real-time in-
app customer support, Changelogs, and Status Pages are just some of the things we
talk about in this book. We’ve seen time and again that a commitment to doing these
things properly drastically raises the bar.
Those first two levels are what is required for a Minimum Viable Product. This is
the point where people will start paying for your stuff:
Reliability: Do the servers stay up all the time? Will the startup building the product
still be around 6 months? Will it be able to handle the pressure of our high traffic
usage? Yes? Whoohoo! Super. Next.
Next up is setting the stage for Product Success. Are you fostering an environment
where users where users are giving you honest, thorough feedback on their
experiences? This is the point where products start to turn the corner:
Communication: Does the product communicate when it’s going down and why? Is
it easy to send in support requests, and do I get timely responses that instill a sense
of trust?
Community: Do I feel like there are people on the other end of this app that actually
give a crap? Do I know any of their names?
First, the wording and the thoughts here are new. We’re not exactly seasoned
authors over here at Ramen. At times, the writing might be a little long-winded or too
technical or even contradictory. We’ve tried our best to mitigate this, but we’re not
perfect. To that end, you’re going to see these little icons all over the book. These
are links, and you can click them. They link to a feedback page on our website where
you can leave comments about that part of the book. We’ve never seen anyone else
do this, but we think it will help us learn from you how we can improve this and
future books.
Third, we are huge hypocrites. If you go through this eBook and then you look at all
the things we build, you will almost certainly find instances where we don’t follow
our own best practices. All I can say in response is that running a product is a lot of
work, and there are a LOT of things that any company could be doing at any one
point. You have to prioritize, and that means that at least few balls are always going
to be dropped.
This book is not meant to be a clarion call for everyone to stop what they’re doing
and implement these best practices in their next sprint. It is simply meant to give you
something to think about as you struggle with something that all Product Creators
struggle deeply with: getting a better understanding of what customers need and
want in the pursuit of creating successful products.
Read on, and let us know what you think. We’re looking forward to hearing from you!
The only real “requirement" is that you’re working on building a SaaS product.
That doesn’t mean you have to be at a SaaS company. You could be at a very large
company where a SaaS product is just a small part of the overall product line or you
could be working on an internal tool that’s only going to be used by a hundred
coworkers. Regardless, these best practices can still help you create a successful
product more efficiently.
Beyond working on a SaaS product, there are really no limitations on who can
benefit from this book. At times, we will specifically address Product Creators:
people who have a significant role in the decision-making process when it comes to
figuring out what goes into a product. However, since responsibilities often blur in
organizations, the more people exposed to these ideas, the better. People in
marketing, engineering, sales, product, or operations can all benefit from this eBook.
The
One thing we love at Ramen is immediate, real-time, contextual feedback. The eBook
format doesn’t really afford us the opportunity to get feedback in that way, and that
drives us nuts.
We started to think about ways we could easily get feedback on this book in order to
improve future editions and future eBooks, and we came up with this: .
Throughout the book, you’re going to see this little icon: . You can click it, and
when you do it’ll take you to a page where you can give feedback on the section of
the eBook you’re reading.
Your end users get a 404 when they go to a non-existent URL, such as
https://ramen.is/a-url-which-will-never-exist-or-will-it. This can happen due to typos
or things being deleted/moved. They get a 500 when there has been some sort of
error in your application.
In addition to 500, there are dozens of other error codes such as 400,
401, 402, 403, 502, 503, and 504. Each of them means something very
specific, but many web apps lump them together under a single, static
error page. For the sake of brevity, we’ll refer to all errors as 500s for
the rest of this article.
Ruby on Rails rocked the error page world when it launched this radically advanced
error page back in 2006:
Title tag: “You’ve ripped a hole in the fabric of the internet. Love, Chris from CSS-Tricks"
Even companies like The North Face got into the game:
Is this an error or a 404 page? Not super clear. Report to the Web Administrator
doesn’t really scream “We care about your experience and want to fix whatever it is
that got you here." This sentiment is reinforced when you click it and find that it’s just
a mailto link. This is a great example about how putting the wrong kind of effort into
a 404 page can actually be a net negative.
Error pages should clearly state that something is missing or broken (Tweet this). To do
otherwise, even for the noble goal of Goat Shenanigan Awareness, risks confusing
your customers.
But for SaaS products, these pages are a big deal. Your customers might have been
mid-workflow. This might have been the one time a month they click on that button
to get whatever it is they pay you $5,000/month for. They click “Send" and get an
error page. Not cool. If you’ve spent any time making a funny 500 page, at that
moment your customer might not share your sense of humor.
Not only does an error page mean your customer was unsuccessful at finding or
doing something, but sometimes you don’t even know it’s happening.
Can you believe that guy? Don't listen to the Yeti. #FightTheYeti
Even if you have exception monitoring in place, most of them don’t record 404s. I
once learned this the hard way, because it turns out that it’s a super big deal when
your billing page is at /manage/billing but your navigation links to /billing.
#facepalm
All that being said, it’s still possible that you will miss errors. There are multiple layers
of software between the Internet and your web application (eg. nginx, varnish,
unicorn), and it’s possible for any layer to throw various types of errors even when
there hasn’t been an “error" as far as your application is concerned.
Your error pages should ask your customers how they got there and what they were
expecting. Your error pages should apologize for the inconvenience. If it’s the truth,
In addition to asking questions, a custom solution also gives you the flexibility to
store additional details like exception data and session information to aid in
debugging.
One thing to keep in mind, though, is that to work correctly, this system must be
isolated from your main application in the event of a cataclysmic failure. That means
using a different application, database, and should probably even be hosted on a
separate server. That’s a lot of work, and I told you earlier this was Low to Medium
on the Initial Effort scale, so below, I list three 3rd-party services you can use on your
error pages. Each of them have generous free usage tiers, and can be setup in ~5
minutes.
Google Forms is free forever, and provides a simple way to add a questionnaire form
directly to your 500 pages. I was able to create a new form and add it to Ramen’s
error page in about 4 minutes and 30 seconds. Not kidding. Here’s the video sped up
4x.
Method 2: Wufoo
Another option is Wufoo, which has a generous free tier and integrates with many
other services. It took me under 5 minutes and 30 seconds to create an account and
have a form live on Ramen’s error page. Here’s that video sped up 4x.
Method 3: Ramen
Not all chapters in this book will promote Ramen, but for this best practice, Ramen is
an option. We have pre-canned 400 & 500 questions that you can use, and some
extra goodies that help managing responses at scale a breeze. Video here.
SPAs are built differently than ‘standard’ web apps, but everything we talked about
above still applies. However, since most SPA errors are asynchronous, there’s not
really any error page to show.
Normally, SPAs are built to catch errors on their asynchronous requests and display
error messages accordingly, but errors still slip through the cracks. And if anything,
it’s those errors that will be most confusing & frustrating for your customers. In this
case, it’s still possible to use the above products to ask your customers questions
In the wild
We looked around for companies that do this well to include as case studies, but we
had a tough time finding any. We believe this is for 3 reasons:
If you have built or used a SaaS product that has great error pages, please let us
know: .
I understand that point of view. Running a product is an exercise in herding cats, and
it takes a while to get to everything you want to do. However, as I’ve said a few times
before, Product Success is about what happens at the periphery of your product, as
it’s in those moments that customers face crossroads: do they come to trust you
more, or do they get annoyed and think about sending out a mean tweet.
E-mail receipts are virtually no effort and have huge upside for SaaS products.
(Tweet this)
From a customer service point of view, they did great, but let’s look at it from a
Product Success point of view by breaking down the end user experience:
But that took like 10 minutes of my life. Despite customer support doing a great job,
they did not make my life easier. They made it harder.
Sending an e-mail receipt would have prevented this problem from ever existing. I
would have gotten an email a couple months back, replied to it saying “whoops can I
get a refund?", and gone about my merry way.
To use a biological analogy, e-mail receipts are like germs: they strengthen your
“immune system" by making you deal with people churning who aren’t getting a ton
of value out of your product. E-mail receipts make you stronger.
Can you believe that guy? Don't listen to the Yeti. #FightTheYeti
Product Fail Yeti raises an interesting point. When you get into the thousands of
paying customers, and have raised tens of millions of dollars, those board meetings
can get pretty intense. Is not sending e-mail receipts an oversight? Or is there
concern that starting to send receipts after not sending them may result in a jump in
churn due to customers canceling once they realize they’re paying for something
they’re not using?
Not joking. We’ve seen email receipts before that didn’t actually say who they came
from.
People often have multiple accounts on a single service. Make sure to include some
kind of account # or other unique identifier.
3) Include the original amount, any discounts, and the final amount
If you went through the trouble of giving someone a discount, remind them of it!
You should never send emails with a no-reply@ email address, and especially not
when it comes to receipts.
Include the type (Visa/Amex/MC) and last 4 digits on the card. If you have the name
on the card, send that along too, as it can help avoid some headaches.
6) Explain credits
If there is a credit on the account that is being charged against, make that clear. If
there is a clear date where those credits will be out, say it in the receipt. For example:
“Your credits will last 3 more months. After that, we’ll start charging your card on file:
Visa 5344"
When it comes to charging overages, no amount of detail is too much. If you need
the email to link out to a web page to show all the details, that’s fine, but make it
clear that there were overages and let your customers know you’re able to describe
them.
When it come to renewing, e.g. in the case of Annual Billing, you should give your
customers ample notice. This is a great opportunity to learn from a customer. Don’t
fear that they might churn when confronted with the fact they’re about to get
charged again. As I said above, it’ll only make you stronger.
Tools
This week is pretty simple. When you charge someone, send an email. It’s basically
free. It can be an ugly text email or a beautifully-designed, responsive HTML email
like this Mailgun design:
In the Wild
Keen.io
I mentioned Stripe above, and Keen uses it. Here is an example receipt from Keen
automatically sent through Stripe:
Slack
Slack does annual billing, and they send you emails when credits are added to your
account:
• http://www.getvero.com/resources/dollar-shave-club/
• https://litmus.com/blog/best-practices-for-optimizing-order-confirmation-
emails
Have you come across any companies that have phenomenal email receipts? We’d
love to hear about them for a future edition of this book:
Many people have different reasons to care about a great About Page for their
company:
But Product Creators have reasons to care about the About Page too.
In today’s article, we’re going to talk about why you should care about the About
Page, refute common justifications for not having one, explore some best practices
that even companies with existing About Pages can use to improve them, and
expand on how larger companies with many products can use About Pages at scale.
When I look at an About Page, especially for a SaaS product, I’m looking for a few
things:
• How big is the team? This gives a sense for the maturity level of the company.
• What are the titles given to people? This gives a little information on the
company’s priorities.
• Do the employees have Twitter handles? I like to see if their personalities
come through.
• Do they have any big name customers, investors, or partners? Knowing they
have large partners lowers the chance they’re going to fail.
There’s a lot to unpack in that list, but to put it simply, I can usually tell within 30
seconds of looking at an About Page if anyone at the company gives a shit about
their product. If I come away with a “Nope", that’s pretty much the ballgame as far
whether or not I’m going to sign up for their product.
But, when I come away with a “Yup", I get a real kick out of connecting with those
people, whether it’s following them on Twitter, shooting them a personal email, or
just making a mental note to engage with them or their company if the opportunity
presents itself.
I’m not alone in yearning for an About Page when first looking at a new product. Just
look at what I was able to dig up in less than 5 minutes of searching on Twitter:
Amy Ellis, a postgraduate digital publishing student at Oxford Brookes, says she
won’t bother signing up for any service that doesn’t have an about page:
John Sheehan, CEO & co-founder of Runscope, agrees with Wade Foster, CEO &
founder of Zapier, that Breezy appears to have a great site, but still takes time to call
out their lack of an About Page above all else.
Having a great About Page will increase your customer's trust in you, which will in
turn make them be more amenable to giving you real insight into what they need in
your product.
There is no downside
Here are some things I’ve heard people say as reasons to not have an about page,
followed by a rebuttal. Feel free to memorize these refutations and recite them as
needed.
Let’s pop up a few levels here: if all it takes is a recruiter to know your employees’
names and poof their gone, you have much bigger problems to worry about.
“Some new hires are very strategic, and publicizing them could expose our
strategy before we’re ready"
As someone that has never made such a hire, I can’t completely discount this point
of view. However, I will point out that this is a reason to not have a specific person on
the About Page, not a reason to not have an About Page.
Really? Take your home page, remove all the copy, add some photos. You can also
use the tools mentioned below like Tumblr or Twitter Lists to jumpstart a lazy About
Page in seconds.
Also, if it really takes a lot of effort to keep your About Page up to date, then you’re
either growing massively, in which case you probably can afford to have a designer &
a developer team up for a day to build a scalable solution, or your company has
some serious turnover issues that should probably be addressed before working on
anything else.
Just ask yourself if you’ve ever checked an About Page before signing up for a new
service. I’ve never met anyone who can answer No to that. If you can answer No,
then please say so in the comments. I’d love to discuss it with you.
Above are some stats for “Goal 5," which is the goal associated with creating a
Ramen account. I created a segment called “About Page," which only includes visitors
who have visited our About Page. In this case, we’re looking at a 5x increase in
conversion rate between those two segments. I know that correlation does not imply
causation, but looking at these data is still interesting. We’re left with three ways to
interpret:
However you interpret these numbers, the bottom line is people who visit our About
Page end up signing up for Ramen at a rate 5x that of people who don’t. Sounds like
a page that deserves some love, no?
Can you believe that guy? Don't listen to the Yeti. #FightTheYeti
When you’re all excited, it’s easy to conflate these. Separate them and make
everyone’s life easier:
• Tell us why you started this company and what problems drove you up a wall.
• Tell us what your products are and what they do.
• Tell us who the people are behind this vision and these products.
Not only does Palantir do a phenomenal job at this, they actually tell you that they’re
doing it:
Vision is one thing, but giving your customers a sense to your greater story can go a
long way towards framing their experiences. Moz does a phenomenal job telling
their story. Starting with the founding of the company by Rand and Gillian (Rand’s
Telling the story can also help to set expectations. If you’re 10 years old, like Moz,
that history gives you a lot of credibility with larger companies. Those companies
immediately understand that you know how to support them and that you’ll be
around a while. Being young can be an asset as well. Being upfront about your
company’s youth can attract customers looking to be early adopters of new
technologies. These early adopters, by their very nature, won’t mind giving you some
leeway as you come up against growing pains.
Got some big name customers, investors, or partners? Don’t be afraid to show them
off if you want to. Social Proof gets a bad rap in some circles, but in my opinion it’s
perfectly fine to list them out, as long as mentioning these 3rd parties takes a clear
back seat to telling people about you. In this example, Segment even goes so far as
to link to their open source software projects, a great way to get street cred with a
technically-minded customer base.
A “Metalsmith Socrates Ripple Nightmare" sounds terrifying, but that’s just Segment-slang for their secret puppy room.
The rest is up to you. On the About Page for Jirafe–pronounced Giraffe–you can filter
their employees based on attributes like whether or not they are Falafel Enthusiasts
or a Giraffe.
If you’re someone like Hootsuite with over 1,000 employees, an About Page not only
becomes unwieldy (and probably loads very slowly if you’re including headshots) but
it also loses its ability to cleanly convey who the people are behind the different
products.
This led us to ask ourselves: Why do larger SaaS companies tend to limit themselves
to a single About Page? (Tweet this)
There’s no reason that products can’t have their own About Pages. We haven’t seen
this, but I can tell you without a doubt that if I saw a link in the header or footer with
the text “About the Product Team", I would click on it and enjoy learning about the
people behind the product.
Tumblr
Your About Page could just be a Tumblr where each person writes their own entry.
Using Tumblr’s free subdomains, you could even have about.yoursite.com.
Twitter Lists
You could create a list under your company’s Twitter handle and link to that list, like
this.
AngelList
You can link directly to the Team section of your AngelList profile using the #team
anchor. For example: https://angel.co/ramen#team.
Further Reading
Here are some other posts that will whet your About page whistle:
• http://www.copyblogger.com/how-to-write-an-about-page/
• http://blog.hubspot.com/marketing/remarkable-about-us-page-examples
• http://www.clickz.com/clickz/column/1691484/the-power-about-us-page
• https://www.rocketspark.com/blog/why-about-us-page-so-important/
• https://www.quora.com/What-are-some-of-the-best-company-About-pages
Do you have a favorite about page or have questions? Reach out to us and let us
know!
We’re talking about status pages today because every single SaaS product should
have one. Status pages have become pretty common, but their not perfect yet. This
post will explore why you should have a status page, and how to get one quickly set
up with the goal of giving you the ammo you need to finally set one up.
First, and most obviously, status pages are great because software it super
complicated, and it breaks in really annoying ways all the time. A ‘simple’ SaaS
product today can rely on dozens of 3rd party services. Some of those 3rd-party
services are critical path, and if they go down, the app goes down. SaaS products are
either in the cloud, or in the case of native mobile apps, rely on APIs in the cloud.
Clouds break too.
For a SaaS customer who might be logging in for that one time a month to push the
button that gives them whatever it is they pay you $5k/month for, telling them “Hey
we’re down because of XYZ right now. Hope to be back in 30 minutes" is much
preferable to having them sit at their computer freaking out thinking their Internet
connection is down for half their morning.
Some product people we spoke to worry that status pages can decrease customers
confidence because it highlights mistakes. We’ve seen the exact opposite. Status
pages demonstrate to customers that you care. How you describe disruptions, their
root cause, the fix, and the steps you’re taking to ensure they don’t occur again is
one of the best ways we know of to gain your customers trust.
Furthermore, status pages help to manage expectations. Software is not perfect, and
nobody should be under any delusions to the contrary. Having a status page says to
your customers “look, things are going to break. it happens. but when it does, you
can see by our track record that we bend over backwards to make things right."
Can you believe that guy? Don't listen to the Yeti. #FightTheYeti
Check out this Twitter search for ‘but “your status page"’:
Diving into one of the tweets, we see an interaction between Intercom and one of
their customers:
Nobody wants their customers to be doing QA, but having them help you out from time to time is a plus.
And if we click the stspg.io/1YM5 link that Intercom tweeted right after that, we see:
So it turns out that the incident on their status page was created because of a
customer report on Twitter. Pretty cool.
I don’t think Chris is canceling his Intercom subscription any time soon :)
Status Pages not only increase trust, but increase communication between customer
and product team. If you don’t have one, get one :)
Whenever there is an incident, create an entry. Be sure to include the time it started,
an explanation of how it is manifesting and where (eg. “intermittent 502s on web
app" or “API completely down").
As the incident matures, edit that entry with updates on what is happening, taking
care to include timestamps of each update and any available ETA.
When the incident is resolved, mark it as such, either using a tag or putting
“[RESOLVED]" at the start of the title.
Use your status page in combination with some kind of in-app notification or mailing
to notify customers of impending downtime. This could be when upgrading
hardware that will result in known downtime, or software updates that are larger
than usual and should go smooth but might not.
Some companies will send out emails to their entire customer base when there are
huge outages. This is awesome, but be careful to not let all that work writing your
Tie your status pages incidents into a system that proactively notifies customers
when things happen. These messages can go out via e-mail, SMS, or even push
notification if you have a native mobile application.
This obviously has it’s limits. If your product has 400k customers, 20k of which log
into the web-app per month, 200k of which are idle, and 180k of which simply get an
email report from you once a day, you don’t really wanna email 400,000 people if
there’s a hiccup in the website as it doesn’t affect all of them.
I’m not proposing a solution for how to manage the above case, but I
wanted to call to attention that some of the things we’re talking about
here are a lot more nuanced than they may seem at first.
5. In-app integration
I hadn’t actually seen this until Freckle, but they put a little dot right in their app. This
is awesome, because it removes the need for a customer to through the following
exercise:
6. Oopses
We don’t see this one much, but status pages don’t have to just be for service
disruptions or outages. Every now and then, people screw up. A mailing goes out
that wasn’t supposed, or someone tweeted something stupid from their phone
forgetting their Twitter client was on the company account.
Tumblr is a great, free solution for a status page. Their infrastructure is stable,
redundant, and can handle major traffic. People can ask you questions and
comment on posts as well. We use Tumblr for Ramen’s status page, and we just
change the color of the title text whenever we need to. They even support custom
domains, so you can still have that status subdomain like we have for
http://status.ramen.is. They do not, however, support SSL, so if customers stumble
across https://status.ramen.is, for example, they will think you have no status page.
WordPress
WordPress is also a great, free option. One advantage WordPress has over Tumblr is
that people can subscribe to email notifications on new incidents. You’ll need to be
mindful of how you deploy WordPress. If you deploy it on your own infrastructure, it
could go down along with your application during a service disruption.
WordPress.com provides free hosting, but if you want a custom domain, you’ll have
to pay.
You’ll also want to be careful to make sure your installation has enough horsepower
behind it. The thing about a status page is that it requires zero resources until
something goes wrong, and then it needs to handle massive traffic. If you go this
route, and do a self install, do yourself a favor and setup a CDN & page caching with
plugins like W3 Total Cache.
Stashboard
Stashboard is a project that came out of Twilio in 2010, and has cemented itself as
the go-to open source purpose-built option for status pages. It’s built to run on
Google App Engine, so there is a generous free usage tier, and unless your
application also runs on GAE, it’ll stay up even when your entire hosting
infrastructure goes down.
StatusHub.io
StatusPage.io
StatusPage.io has cornered the market on hosted status pages over the past year.
With a keen, product-focused eye, they’ve built a great product that addresses all the
rough edge cases that make status pages a lot more complicated to implement than
appears at first glance.
They don’t have a free tier (although you can play around with their product and
view your status page privately for free), and their cheapest pricing tier is $29/month.
This is a steal if you have any revenue at all, but the lack of a free tier means you
might want to start out with one of the free options above until you grow a bit more.
At scale (re: Enterprise) they have a bunch of great solutions for private status pages
as well, which can be important if the application your reporting status on is an
internal-only tool.
Disclaimer: The Status Page people have an awesome product and are
personally awesome. We know this because they work down the street
from us. However, that had no impact on being listed here.
In the Wild
There are a ton of great companies using status pages including:
If you don’t have a status page for your SaaS product, I hope we’ve convinced you to
join the ranks of these great companies.
Have a great example of a Status Page or have more questions? Shoot us a note:
Changelogs don’t have to be that nerdy (Tweet this), however. In SaaS world, many
successful products like CustomerIO and Intercom have taken the idea of a
Changelog and applied it their products. They regularly publish updates on the
changes they make, why they made the change, and explain how the change will
affect customers.
Anyone who has ever produced content can look at the example above and
immediately tell that these are not quick posts. They take a lot of time. These are full
blown blog posts with collateral and editing.
Why do companies spend their time on this? Why do we agree that they should?
What good is fixing a bug if nobody knows about it because they’ve been avoiding
“that buggy part of the app"? What good is a new feature if your existing customers
don’t even know it’s there?
And they’re right. Changelogs are a part of Relationship Marketing. (Tweet this)
How many times have you came to a new SaaS product website, saw their last blog
post was 6 months ago and their last tweet was 4 months ago, and thought to
yourself “NOPE"?
Customers like knowing the lights are on even if the things being changed are not
something they’d use.
Can you believe that guy? Don't listen to the Yeti. #FightTheYeti
For people like myself, an awesome changelog can make someone fall in love with a
product before even using it. Changelogs are an opportunity to explain how you
think about your product. (Tweet this) It might not be a sexy marketing tag line that’ll
convert you a billion users at a 90% conversion rate in 3 days, but it’ll make the
astute reader respect the hell out of you, your team, and your product.
Intercom’s changelog includes thoughts on how and why one might want to use the new feature.
It’s a sign of maturity when a company is thinking about what they’re changing in
discrete chunks. CustomerIO recently released version 5.5.12. Ok. These guys aren’t
some young pups. They’re not screwing around. They are rigorous. I trust them.
“Oh really? XYZ is crap and doesn’t work in IE at all" - Customer comment
Just like email receipts are awesome because they make your customers ask
themselves “do I really need to pay for this?" (which, trust me, you really do want
happening) Changelogs are awesome because they make your customers think “oh
Finally, we circle back the definition of Product Success: Product Success is about
being thoughtful when it comes to what a product does at the periphery — outside
the core product — and how those peripheral interactions foster an environment
through which many direct and indirect forces carefully nudge the product to get
better and better over time.
Knowtify.io
Knowtify has a bunch of different tools, but one of them, called Inbox, can be used
for new feature notifications: http://www.knowtify.io/inbox
Ramen
We started Ramen with the idea to help SaaS product people be successful, and as
part of that we built something we call “Product Center". Product center allows you
to share product ideas and updates, as well as getting your customers to comment
on them. You can try out Ramen for free.
Have you seen a great example of a Changelog, or do you have more questions
about setting up your own Changelogs? Ask away:
If you’re reading this in 2015, and you just you took a break to go Googlin’ for “LRQ",
you’re probably not going to find much. That’s because we coined this term internally
at Ramen and this is the first time we’re using it publicly.
An LRQ is a single question that you ask your end users, either in-app or via email, as
part of the natural course of their usage of your product. LRQs all share a few
characteristics:
You start off by picking two interval periods: initial and ongoing.
1. After initial days, you ask your customers the following question:
"How likely is it that you would recommend Ramen to a friend
or colleague?
2. After answering, the customer is usually prompted to “Tell us
more about why you chose X" in a follow up question, but that’s
not required.
3. Every ongoing days after that, you ask your customers the exact
same question.
4. Every day, take all the responses–ignoring any people who
dismissed the survey without responding–and do the following
math:
◦ If they answer 9 or 10, label them as a Promoter.
◦ If they answer 0 through 6, label them as a Detractor.
◦ If they answer 7 or 8, label them as a Passive.
5. Get the % Promoters by dividing the # of Promoters by the total
# of responses
6. Get the % Detractors by dividing the # of Detractors by the total
# of responses
7. Subtract the % Detractors from the % of Promoters, remove the
% sign, and you’re left with a plain old scalar number between
-100 and 100. This is your Net Promoter Score.
Net Promoter Score is just that: a score. It’s not a percent. There are not passing and
failing grades. 60 isn’t necessarily good and -10 isn’t necessarily bad. People who talk
about “needing to bump their NPS up a few percent" are not only using the wrong
terminology, but they’re probably thinking about it all wrong to begin with.
What’s really important with NPS is how it changes over time. See the NPS start to
tick up recently? You’ve probably been doing something good.
A new marketing initiative that goes after a brand new persona, for
example, could cause a flood of new, really excited customers into your
product. But if, 45 days later, it turns out that the language in the
marketing copy incorrectly led people to believe that you had a certain
set of functionality that you didn’t, they’re going to churn at very high
rates. In this pathological example, you could have changed nothing in
your product, and still over the course of 8 weeks seen a spike in NPS
followed by a spike in churn.
Opinions on NPS aside, LRQs are a vital tool for product creators
Whether you love or hate NPS, it’s important to remember that what really matters is
that you have some set of LRQs to act as a proxy on the pulse of your customers’
attitude towards your product. Every SaaS product should have at least one (but we
could make an easy argument for three) active LRQs that they look at every week.
What matters is that you regularly look at the results and start to get a “feeling" for
the data.
Before we get into what types of LRQs you should be asking, I want to talk about two
distinct “families" of LRQs: one-time and periodic.
One-time LRQs are asked transactionally. (Tweet this) Examples include right after a
user signs up, after they’ve been a customer for a certain amount of time, or after
Periodic LRQs are asked repeatedly (Tweet this) They are asked initially after a certain
amount of time, and then at a regular interval thereafter. A valuable characteristic of
periodic LRQs is that you can monitor for customers that change their answers over
time. NPS is a periodic LRQ.
Because of the repetitive nature of Periodic LRQs, there is a lot of nuance to how
they are implemented.
They need to be asked in a rolling fashion, so you don’t have a “survey day" where
you ask everyone that logs in on a certain day the question. This opens you up to
several biases. If the majority of your customers are in SF, and the Giants lose an
NLCS game in extra innings late last night, your customers might not be in a super
good mood, and that can really affect your numbers.
Unlike one-time LRQs, periodic LRQs are rarely asked of brand new customers, as
the context of their answer will dramatically change by the time they answer the
second time. You want to give them time to reach a “steady state" or else you’ll
always see huge changes in their answers once they start getting the question for
the second time.
Let’s say you want to establish an LRQ to monitor the “health" of your checkout
process, and you set it up so that the question will appear after your customers
finishing buying something. You don’t want to survey them every single time, so you
set up the question to only be asked if it’s been 60 days since the last time they were
asked.
However, in this scenario, you’re only asking people the question after they’ve
successfully completed a transaction. Maybe your process isn’t actually very good.
Maybe your workflow is so confusing that people abandon in many cases. I’m
painting a pathological picture here, but it illustrates the point that randomly asking
customers who have completed a checkout process:
Hi Matt, Over the past few months, have you found our checkout process to
be a satisfying experience?
Is much different than asking them right after they finishing buying something.
Example LRQs
Ok, if you’re still reading, bravo. This is what you’ve been waiting for. Below, we’re
going to go over some LRQs that you can start asking right now, and follow up with
some tools to help you do it.
This is a tough one. For most SaaS apps, in-app gets you a lot further, as questions
can be asked in context. This allows them to be shorter. It also affords much higher
conversion rates. We’ve seen in-app response rates around 40%, where email
response rates are ~ 15%.
However, many products have usage patterns that make in-app not super useful.
These include situations where your customers don’t interact with the product in
their browser frequently, but maybe simply enjoy a daily or weekly email. In this
case, you probably want to lean towards email questions.
Once you start asking these questions, you have to do something with all those
amazing data. You’re technically inconveniencing your customers by asking them
these questions, so make that worthwhile.
Get into a regular cadence of reviewing answers to questions. Get a weekly email
setup that sends you the following for all your LRQs:
Obviously you want to see the most recent answers, and you probably want to see
answers from all-time, but you’re going to also want to compare the most recent
answers with the precedent time period (read: last week vs. two weeks ago)
For periodic LRQs, your weekly report should include the number of people who
have changed their answers since the last time they answered. This is a lot easier
said than done, and in most cases will require some custom development on your
end.
• SurveyGizmo & SurveyMonkey – Both offer free tiers and make it easy to send
surveys via email.
• Delighted & Promoter.io – Very similar products that provide an email-only
NPS solution. However, as of writing this, their free tiers are not very
generous.
In-app
• Wufoo & Google Forms – Wufoo & Google Forms can do a lot for free.
Although the UI of popping up an iFrame w/ a Wufoo/Google form would
leave a lot to be desired, and it would leave you to do all the NPS math on
your own.
• SatisMeter – A very straight-forward tools for implementing in-app NPS. They
have a free tier, but it’s not terribly generous.
• Intercom – Intercom can be shoe-horned to do this, but you’ll have to do all
the math on your own.
• Ramen – Here’s the thing: the lack of a proper solution to do this is pretty
much the reason I started this company. So, I’m not going to belabor the
point: we have NPS and generic LRQ support, and I’ll just leave a little link
here in case you want to try Ramen out. TL;DR: These LRQs are tough to get
right, and as you can probably tell from this extremely long-winded chapter,
our team has spent a lot of time thinking about it.
Can you believe that guy? Don't listen to the Yeti. #FightTheYeti
In the wild
Here are some great examples of companies using LRQs
Mention.com
After reading a draft of this chapter, Guillaume had some great comments which he
gave me permission to post here:
We discovered that you can’t handle answers from your free and paid users
the same way. And if you do, you shouldn’t. Because someone paying you
hundreds of dollars a month and leaves a text comment, that needs to be
handled by a human being.
Never lie to your customer. If he thinks (because you smart marketer let him
believe that) there is a human being reading / replying to his survey, then
you need to be able to handle the volume of answers. We generate north of
1,000 NPS surveys per month at Mention, and though it’s some work, it’s
definitely worth it.
SugarWOD
SugarWOD has a native mobile app for tracking CrossFit workouts. They also have a
web UI for gym owners to schedule workouts and monitor their athletes’ progress. A
question they’ve had in the wild for a while is about getting first impressions:
Drew Larsen, founder & CEO of SugarWOD, highlighted this answer as one of the
most interesting so far:
Drew said that: “Knowingly or not, this guy shared with me his biggest concern /
objection / trial criteria. I followed up with some rollout tips, some stats on what kind
of adoption to expect and created a “to do" to follow-up with him on this specific
topic after a week."
“Hey guys – quick question. Do you see this platform expanding to emulate
services as seen on WodHopper, Wodify, FrontDesk, ZenPlanner, etc?"
Have questions about what types of LRQs you should be asking in your product?
We’re all ears and happy to provide some feedback:
To take that one step further, it’s just as important to design for any screen size
between a small phone to desktop. Screen sizes changes all the time and your
product should be ready to adapt.
We’ll use Trello as an example. Here you can see their dashboard used to manage
projects and organize anything. It would be safe to assume a majority of usage
comes from desktop users.
Surprisingly a lot of the functionality is left in but the UI is simplified for mobile.
Can you believe that guy? Don't listen to the Yeti. #FightTheYeti
First impressions matter more on the web than anywhere else. If your product
doesn’t present a professional look on mobile, your customers will assume your
product isn’t up to snuff. If you’ve made effort to make your website responsive and
delivers the right experience at any screen size your customers will notice.
When designing mobile-first you are forced to focus on core content and
functionality. With limited space you better deliver the functionality your customers
need up-front and easily accessible. When done right this translates to making your
desktop service just as usable if not more so.
Take MailChimp as another example. When you view MailChimp on a mobile device,
you are presented with the information you need and not much more.
When we bump up in size to desktop we add in more features like ordering, mutli-
select options, filtering and other features that are “nice-to-have" but not essential
for mobile users.
When you think about how your customers will use your product and in what
context you focus on what is important to them. You will discover what is really
useful and what features might be left unused, clunky, and not fail to add tangible
value.
Taking a simplified approach to your product and content from the start helps keep
your product lean and flexible. Have you heard the phrase “If I had more time, I
would have written a shorter letter?" For mobile let’s translate that to “If I had more
time, I would show the user less“.
Toolbox
Building your site mobile-first is more about the process than the tools, but here are
some things to get you started.
Implementing a grid system is a relatively easy first step towards being mobile-
friendly if you already have an established website and style set up.
I’ve personally used grid systems like 960.gs and Less Framework 4 to help move
clients into a more standard, structured site. Check out the latest grid systems here
and make your choice wisely.
If you are starting your website from scratch it’s probably in your best interest to
start with a framework that has the bells and whistles for mobile ready for you. Just
remember these are meant as a starting point, any good product will change to fit its
customers needs.
Probably the most widely used Frameworks are Twitter Bootstrap, HTML5
Boilerplate and Foundation. Check out other frameworks here.
Click the green R below to share your experience with mobile development. How has
it helped you improve your customer experience? Did you see improved conversion
or less churn after switching to a mobile-first approach?
For a small team, having people rotate through “in-app chat duty" and then come
together once a month to discuss what they’re seeing is easy. At scale though,
managing in-app chat can be super complicated to get right. This is why it’s
important to not just “throw in-app chat into the app and route them to customer
support." A Chat Czar not only establishes the processes for managing in-app chat at
your company, but they’re the ones who will periodically review chats to keep a
subjective finger on the pulse of what’s coming through chat. This person will usually
just be the Product Creator (or Product Manager, or Owner, etc…) but it’s worth
thinking about this as another title. It’s that important.
2) Take the time to test and choose the right tool for you
Like many things, in-app chat is simple in theory but incredibly complex in practice.
Each tool has nuanced differences that can make a huge difference in how it works
for you, so if you’re not already using one, take the time to try a few out and see
what works for you. There’s no reason you have to do your due diligence in a
vacuum. Install one for two weeks, switch, move on. If you’re a Segment.com
customer, it’s super easy to install and switch between tools like LiveChat,
LuckyOrange, Olark, SnapEngage, Userlike, Intercom, Zopim, and more.
Nothing is worse than sending a message off into the void and wondering when it’s
coming back. Nothing, that is, except getting something back that is ambiguous. How
many of you love hearing “cool I’m gonna get to that soon" when talking to a friend
or coworker? “Soon" can be 5 minutes or a week. Manage expectations by ensuring
that your users understand what timeframe to expect a response in, and what times
you are in the office. Also: Use timezones, people! The world is not flat, and your
users are not at most 3 time zones away anymore.
Everyone loooooves setting up automated responses, but how many of you have
sent a real email or chat to someone and found out later they didn’t respond
because they thought it was automated? Messages can be both personal AND
automated. Let your customers know the difference.
Respond. Even if it’s a “huh. cool just read this. we need to think about this. I’ll set a
reminder to ping you in two weeks once I’ve had a few hallway conversations with
people about this" People don’t mind that. They’d rather that than:
Can you believe that guy? Don't listen to the Yeti. #FightTheYeti
experts in their field. They know the domain of the product inside and out, but a lot
of times customers will interact with your product in ways that are more boilerplate.
We’ve talked about these things before: Status Pages, About Pages, E-mail receipts.
In-app chat is no different.
Simply establishing the fact that you are available makes your customers feel better.
It makes them not just view the product as some pixels on the screen, but as an
extension of the people behind it’s creation: you.
As Product Creators, we don’t need to (and in fact could never possibly) establish
deep relationships with each customer, but we can establish a relationship where
our customers know that “Hey building stuff is a super tough multivariate
optimization problem, but we’re listening, and we deeply care, and we’re doing our
best." Having in-app chat and doing the 7 things above goes a long way towards
conveying that message.
Still not convinced that in-app chat is the way to go, or want to recommend another
company for inclusion in our next edition? Click and let us know:
It’s our hope that you all give us lots of feedback via the little Rs: and we’re able to
put out a second edition or a whole new eBook later this year with more stories,
data, and best practices.
It’s been a blast writing this book. Thanks for reading. See ya’ll on the Internet.
Thanks!
We’d love it if you could share this with friends :)
Share on Facebook
Share on LinkedIn
Ramen
Ramen is a product that helps you learn what your customers need by enabling you
to ask simple, yet highly-targeted questions of them while they use your product.
ChartURL
ChartURL lets you create personalized image charts that can be easily embedded in
emails, Slackbots, PDFs, and more.
Credits
Huge thanks go out to everyone who played a role in this book either by giving
feedback on process, reviewing chapters, or simply providing moral support.
• Nick Sutterer
• Rachel Andrew
• Kimberley Byer
• Mike Lewis
• Steve Klein
• Weston Platter
• Ben Nunez
• The Foundry Group Executive Mailing List
• John Mayfield
• Drew Larsen
• Guillaume Cabane
If you’ve ever asked yourself this question, you’re not alone. We’ve spent years
thinking about this question, and after talking with a lot of very smart people about
how they approach it, we decided to create this blog series to look at this question.
Every week through the end of 2015, we’re going to share and discuss tactical,
scalable best practices for Product Success: a topic that lives at the intersection of
Customer Development, User Experience, Customer Success, Product Management,
and Customer Support.
• Tactical — It will be something you can look at and objectively answer the
question “are we doing that?" with a definitive yes or no.
• Scalable — It will be something that is just as beneficial to a solo founder as it
is to a SaaS Product Creator at a 1,000 person company.
• Hackable — You’ll be able to implement a lot of these best practices by paying
a 3rd party to do the heavy lifting, but there are always hacks, and we’ll make
sure to show you great ways to get things done on a dime.
The first best practice we’re going to explore in this series is Error Pages. In that
post, we’re going to discuss how most Product Creators are missing out on a golden
opportunity when it comes to how they use their 404 & 500 pages to learn from their
customers.
In each post, we will talk about how the best practice is much more than it seems on
the surface, and enumerate ways you can easily get started. As you’ll come to see, a
common theme among many of these best practices is around establishing higher
quality feedback loops (ie. Lean Startup 101: Build -> Measure -> Learn), as raising
the level of discourse between customer and product team is at the core of Product
Success.
If you’re really like me, you then sat at your desk and looked at an empty whiteboard
for a few minutes before cracking open a new browser tab:
I can’t believe what a bunch of nerds we are. We’re looking up “money laundering" in a dictionary.
-Peter Gibbons in ‘Office Space’.
The results you get back contain references to things like Lean, Agile, and Customer
Development among others. These approaches are based on the scientific method,
thereby making them virtually unassailable methodologies from a logical
perspective. Their greatest strength — that they can be applied to any department in
any business — is also a weakness.
This general applicability means that the body of work around these topics tends to
be fairly strategic. Once you wrap your brain around the methodologies, many of us
are still left wondering, “So… what do I do now?"
Creating Categories
As the size of the software industry expands, what were once edge cases for a few
become the status quo for many. As this happens, bodies of work start to pop up
exploring these edge-cases-turned-status-quo in ever-increasing depth until they
eventually reach the stage of Category.
The image below is a screen capture of Google Trends for a few Categories that are
familiar to you. It’s amazing to think that 5 years before writing this, nobody was
talking about “Social Media Management" or “Customer Success". Even more
shocking is the absence of “Growth Hacking" until late 2012.
You think that’s crazy? The iPhone didn’t exist 400 weeks before writing this post. That is still B-A-N-A-N-A-S to me.
This doesn’t mean people weren’t managing their social media before 2010, or that
people weren’t coming up with clever ways to hack growth before 2012. People just
weren’t labeling these activities as such yet. Eventually, a critical mass is reached, and
these things were labeled. The labeling made these categories something to be
focused on, discussed, debated, and refined.
Over the past couple years, we’ve all seen this happen in categories like User
Experience, Growth Hacking, Influencer Marketing, Content Marketing, etc…. There
are thousands of blog posts and hundreds of books that tactically address the
question “So… what do I do now?" for all of those categories.
There is no simple checklist that, if followed, will ensure a successful product. But
there is no simple checklist that will ensure Customer Success or a good User
Experience either, and that doesn’t stop people from talking about best practices in
those categories.
This book is about best practices for a new category: Product Success.
What we’ve been missing is the proper treatment of what happens at the edges of a
product. The Product Periphery.
Product Success happens in the places where you have an opportunity to interact
with your customers in important ways. These can be seemingly-boring things that
are easily ignored: your About Page, a Status Page, your email receipts. And they can
be eyebrow-raisingly odd: designing your product to be mobile first, using things
called LRQs and SRQs.
But those things--those Product Success best practices--can make the difference
between a good product, and perennially awesome product.