You felt that your last week’s email campaign was superb – until you looked at your metrics. How can the weekly MRR growth rate show 0% growth? What’s going on?
It may be that your gut is right and you were deceived by rolling metrics.
You are in a risk if you track SaaS business performance week-to-week and want to use that data to make business decisions.
I get a lot of questions about metrics. Whenever I get an email where someone is really confused about their metrics, it’s almost always because of rolling metrics. This article explains what rolling metrics are, what they are good for and what they cannot be used for. You’ll also see how they can mislead you.
What are rolling metrics?
All SaaS metrics can be calculated using different time periods. Normally the calculation period is 1 month.
But that’s not fine. Your business is online and you can track everything realtime – so you want to see your metrics now and not at the end of the month.
Rolling (or sliding window) metrics solve this problem by keeping the calculation period identical, but changing the days that are included.
Let’s imagine we are calculating 4-week rolling revenue. The number in Week 4 includes the revenue from weeks 1-4. In the Week 5, rolling revenue includes the revenue from weeks 2-5.
￼I’m using weeks in my examples, but rolling month calculations normally use 30 days as their time period. That’s used every month, even for months that have 28 or 31 days.
What are rolling metrics good for?
30-day rolling metrics are used because:
You’ll get numbers faster – every day or minute if you want
Rolling metrics & rolling averages make it easier to notice trends
As a result, rolling metrics are superb at showing if your metrics are trending up or down on a monthly level.
But this comes with a huge cost of losing all insights from shorter periods.
What are rolling metrics bad for?
30-day rolling metrics deceive people because:
Accurate numbers from shorter periods disappear
You cannot compare rolling metrics using a shorter period than they were calculated with
As a result, you cannot use 30-day rolling metrics to find out if one week or day was better than the other. It’s all smooth!
Rolling metrics behave just like rolling averages. They smooth out any spikes and dips in the data.
That’s why we can always use 30-day period in rolling calculations. If Stripe charges several day’s worth of invoices on one day or if the system is down one day, the period is long enough to smooth that down.
This is how rolling metrics can deceive you
In the image below, the green bars show the real weekly revenue. The blue line shows the 4-week rolling revenue.
I’ve marked up some rolling growth rates for you. Just look at the week 6. That was a really bad week – but the rolling number does not show any change. And the week 8 definitely wasn’t any better than week 7.
This happens because when the calculation period moves, it doesn’t only take in a new week, it also drops away one week. So the performance in the week that was dropped out also affects the end result. In effect, all rolling metrics behave like averages.
This is fine, if you are trying to find out if you are doing better or worse in the long run.
This is not fine, if you need to know which week’s ad campaign was the most effective.
If you want to match your business performance to your actions on weekly level, ditch the rolling figures.
How do I know if I have a rolling metric or a regular metric?
That’s something your analytics app provider should clearly tell you.
I currently serve rolling metrics only in the weekly follow-up report. FirstOfficer.io is for strategic planning and trouble-shooting, so you must be able see the exact numbers – rolling metrics aren’t a good fit for that purpose.
When you look at your SaaS metrics, you are probably wondering: “Is this a good value? Am I kicking ass or throwing away money?”
As you only see your own metrics and those that friendly business owners share, it can sometimes be hard to answer to those questions.
Since I launched FirstOfficer.io, I’ve been analyzing a lot of SaaS metrics. Every now and then I reach out to a customer and ask if I could analyze their metrics. Obviously that’s something I can’t keep on doing for long – but it has given me an opportunity to see a lot of different metrics.
In this post, I’ll explain how to read and use Average Revenue Per User (ARPU) and how optimal trend looks in charts. If ARPU is new to you, I’ve written a short information sheet on ARPU for you.
Is $20 a good number or a bad number?
ARPU doesn’t have any optimal value.
But there’s a rule. If ARPU is small, you need a huge amount of customers. If ARPU is big, you can manage with less.
So when you evaluate ARPU, the biggest question is: “how many customers can you reach and acquire?”
If your ARPU is $20 then you need 500 customers to make $10,000 per month.
But if ARPU would be $80 then you’d only need 125 customers to make $10,000 – and if you’d get 500 customers that would mean $40,000 per month.
For me, ARPU is the best metric to guess who the product is targeted to. Low-ARPU businesses often target consumers. If you hit it big, 500 customers is nothing.
For business owner, ARPU is a great tool to evaluate if you have the resources to acquire the needed customers and make your business a success.
If you want to compare your ARPU to another company’s ARPU, make sure that they target the same audience with a similar product concept.
ARPU can give you a hint if you are pricing right
ARPU tells you what plans people prefer.
If ARPU is high related to your plan prices, most of your customers are in high-priced plans. This is a good thing – but it can also be a sign that you are throwing away money.
Why? If big part of your customers choose the highest plan, there’s a possibility that at least part of them would be happy to pay more. And if you aren’t offering them a possibility to pay more, you lose that money.
So when the ARPU hits your middle plan price, it’s time to add new price points or add-ons.
If ARPU is near to your lowest plan price, it may be a sign that you could improve your marketing.
You are acquiring people who pay little, when your product could also serve people who’d pay more. Or, it may be a sign that your product is best suited for the people in lower priced plans.
Always do this type of analysis in the context of your business. Never look at just one metric and make this type of conclusions only from it!
Low ARPU means that most of your customers are in the low-priced plans. So if you try to raise ARPU by dropping the lowest price point, it will most likely backfire.
Optimal ARPU – The sales team effect
When you collect ARPUs from several months and put the values to a timeline, you’ll want the line to go upwards. But the optimal line doesn’t just go up. It looks like this:
I call this ‘the sales team effect’, because if often happens when you hire a sales team.
That’s when people rethink the prices and find out who their ideal customers are. Then the sales team starts targeting those people, often selling high-priced plans with annual commitment.
But nothing stops you from doing those same things without a sales team. Just offer upgrades, add-ons, annual plans and advertise them a little. Everything will happen more slowly without a sales team, but that’s just fine.
Poor performance – the flat line
You might think that the worst ARPU trend is a downwards trend. But it isn’t.
The worst possible ARPU trend is a flat line – because it tells that you don’t have price points or you aren’t experimenting with prices at all.
Having just one plan and not trying out different prices is the best way to throw away money with SaaS. You are ignoring the strongest financial lever that your business has.
Of course some audiences are really sensitive to prices, but maybe you could frame the pricing experiments so that you could roll back if needed?
If your ARPU looks flat, it’s time to do something to those prices!
Downwards ARPU trend is fine when you introduce annual plans
As always with SaaS, long-term business value and short term revenue play a tug of war. If you ask ARPU, the short term wins. But you want long-term wins too, like when you improve retention.
When you push people to the annual plans, ARPU will go down slightly. That’s because people in annual plans often have a discount, so they pay less per month. But as they are committed to stay at least 12 months, they’ll eventually pay you more. Which is great and you shouldn’t panic and start preferring monthly plans again.
As you’ve seen, ARPU is a powerful metric even though it looks so simple. It’s no wonder most SaaS businesses follow it up.
If don’t want to miss out on future articles, drop your email to the box below:
I chose to self-fund FirstOfficer.io because I wanted to have a location independent business. But that’s not the only reason. My fellow bootstrapper Allan compares his accounting SaaS to a cockroach. It’s small, but it’s virtually unkillable – watching as the competitors raise millions of dollars, whirl by and then die.
I put together my favorite bootstrapping resources, so that you can shave off some learning time and build a successful product sooner. I don’t have any financial incentive to recommend these sites and products.
Self-funding = risk management
Self-funding your product is a great way to learn practical business skills in a safe environment.
External funding is not only an accelerator. It’s a financial amplificator that magnifies everything – wins, losses and risks. If you know how to play the game and your personal goals align, it’s a great tool.
But for rest of us, funding is just a risk multiplier.
If you want to start investing in stocks, the recommendation is not to start with loan money. Yet so many people do that for their first startups. Capital is a form of debt, even when you give away equity & power. Eventually, it always translates to money – the VC investors are not stupid. They hedge their bets so that they win whether or not your startup succeeds.
PersonalMBA explains business concepts that you should know. It’s pretty much a compilation of great books, so the greatest value in this book is that you don’t have to read all those other books. The author lists the resources, so that you can dig deeper if you wish.
Starting and sustaining gives you a great overview of what it means in practice to build and run your own SaaS product.
Finding time & money
When a bootstrapper meets another bootstrapper, a common question is “How do you fill the gap?”
It’s also worth to learn to be more money-savvy, but not to the point where you lose the quality of your life. Mr Money Moustache is a nice resource for that, and lots of people enjoy using YouNeedABudget. I just use spreadsheets myself.
Try to think like an investor. Sell less of your time – invest more of it.
Finding a mentor & meeting like-minded people
Bootstrapping is a lonely endurance sport. Having a mentor shortens the time to success.
Big part of my product success is due to what I’ve learn from Amy & Alex in 30×500 - back when it was a 5-month long course. Nowadays it’s different, but I do think they take applications today, so if you want in, hurry! Amy also hosts a BaconBiz conference for bootstrappers.
TropicalMBA have their Dynamite Circle. Dynamite Circle isn’t open for new people all the time so if you want in, prepare for a wait.
They say that you are the average of the five people you spend the most time with. So seize the opportunities to meet other self-funded business owners.
Better yet, put together a mastermind group with 3-5 local fellow bootstrappers. I feel that it really helps to keep up the motivation when you can talk with someone about your business at least once a month, live.
Finding other self-funded businesses
Bootstrappers are active online, so hop in and meet people!
I take part in a mastermind group in Solo.im, and they also have a mailing list and discussions. It’s not super active group, but it gives you access to mastermind groups that might be otherwise hard to find.
Then there’s lifestyle.io that hasn’t really taken off. And GrowthHackers that seems to have a focus in bigger businesses. Joel on Software is a nice forum too, although quite quiet and focuses more on desktop products.
There are also plenty of mailing lists and some of them delivers digests of week’s best articles so that you can find new people and blogs to follow up.
Learning from what others do
Time is a precious asset. If you want to get success in a meaningful time, you can’t afford to learn only from your own mistakes and failures.
Fortunately there are so many blogs to help you! I’m only going to list those that I love so much I read every article in them:
I think Unicornfree is the best place to get started with bootstrapping. Amy is opinionated, but she knows what she’s talking about.
A Smar Bear is probably my all-time favorite business blog. Jason has been on the both sides an built both self-funded and funded successful product businesses. This blog is a goldmine.
Patrick McKenzie, aka patio11, is known and loved by everyone. His articles and emails are full of practical information on how to make your product better. And his blog is huge! Be warned that the posts are long.
LessAccounting Blog always makes me smile. I can read between the lines how Allan loves his life and sees the business as a tool to live your dreams right now. I love that attitude. He writes on several topics, but there’s an index where you can pick just the topics that interest you.
I’ve learned so much from Brennan that I just keep following up him even though big part of his posts/emails are about freelancing. But the business stuff that appears in PlanScope Blog is super valuable.
Jarrod is a fellow 30×500-goer. His book, blog and free email course are god-sent for people bootstrapping their businesses. You don’t want to hire a designer for your baby product, but you don’t want it to look like a roadkill either. As you don’t have time to learn everything, Jarrod points out the essential things that make 80% of the difference.
In addition to these, you can easily find a mass of great blogs and people via the other links in this post.
Podcasts aren’t my thing, but one can’t really skip them either – not when they feature all the interesting people.
“I own every analytics tool out there, and we use them all, but they’re all so opaque it wasn’t at all clear what was going on (or why)”
I’m really glad she shared that, because a lot of entrepreneurs have the same challenge.
You need a bunch of different analytics tools, and you’ll need to know what you can do with each. Otherwise you’ll be looking for insights where none can be found. Or live with a false security that everything is OK while things are getting worse.
So let’s take a look at different analytics tools and their purposes.
The business improvement steps
To master your business and drive it where you want, you do something like this:
Understand what’s going on and where your business is heading. Now you know where to focus and what to improve to get the results you want
Study or troubleshoot the thing you want to improve. Now you have ideas how to improve it
Follow-up so that you’ll know if any improvement happened.
But anyway, there are analytics tools to help you with each of these steps.
Three types of analytics tools
If you roughly group analytics tools by step, it looks like this:
The tools list gives just a couple of examples. In principle, if it tracks user events, it’s probably a troubleshooting tool. If it shows a lot of percentages, it’s probably a follow-up tool.
But none of the tools sits nicely in a single box.
For example KISSmetrics has a simple follow-up dashboard. BareMetrics used to have a view that helped to get a little overview on what’s happening (this don’t seem to be available anymore). FirstOfficer has metrics dashboard too.
Google Analytics could be put to any of these boxes. You can do pretty much anything with it, but unless you are just tracking visitor counts it can be pretty intimidating and has a huge learning curve.
Tools like BareMetrics, GeckoBoard and your own dashboard with several key percentage-based metrics are great for following up that you are making improvement.
If you have a customizable dashboard, you can show just one metric at a time – the one that you are currently trying to improve. That helps focusing on the right thing.
Following up improvements is so important that almost all analytics tools offer some type of dashboard.
What are follow-up tools bad at?
You can’t use follow-up tools to assess your business health and really understand what’s going on. Not unless you already have the big picture of your business in your head and can memorize all the figures that these metrics were calculated from and recall all the relationships between different metrics.
Some top CFOs can do actually that. I can’t do that. There are just too many figures to memorize.
If you try to use these tools as a canary in the coal mine, that won’t work either. They will show you when things are really wrong, but by then shit has already hit the fan.
Why? Because percentage-based metrics have no “this is ok”-range. It always depends on what your overall business situation is.
Let’s take an example:
￼13% of $10.000 is $1.300. But 13% of $100.000 is $13.000, and that might not be easy to replace with incoming customers. So especially if your business is growing fast, you can’t afford to miss up things like that.
Event-based SaaS analytics tools like KISSMetrics and MixPanel are great for troubleshooting and studying your customer-base. But they are a bit complex to get started with.
The most common mistake people do with these tools is to buy them too late.
These tools start to gather events when you buy them and install triggers to your code. They don’t have access to any data before that.
If you play the canary in the coal mine game mentioned previously, you’ll end up buying these tools 6-12 months too late. Then you waste valuable time waiting that you’ll have enough data to actually do something. In the meanwhile, all you can do is to just try out random things. You lose money and may miss your goals.
The gap in troubleshooting tools
There’s one thing that KISSmetrics and MixPanel don’t do well. If you don’t know a specific event or property that you are interested in, these tools don’t help you much. You’ll need guesswork to find out events that matter.
We are seriously lacking good customer success troubleshooting tools.
For example, you can easily find groups of interesting people with FirstOfficer.io, and it would be important to find out which properties and events are common to these people. But that’s not easily done with KISSmetrics or MixPanel.
And that’s why I listed Intercom.io, even if it’s a messaging tool. Intercom.io has an API that can be used to tag interesting people, and they have a view that shows several properties for several people at the same time.
But I haven’t yet found a tool that would take a list of customer emails and return a list of events that are common to given people. If you know a tool for that, please let me know!
Business health & overview tools
This is the area that lacks tools the most – which is why I built FirstOfficer.io.
If you want to understand what’s happening in your business and where things are going, the only other tool that I know of is SaaSOptics.
But if you take a look at their reports, you can easily see that they are targeting larger companies. So if you use accrual-basis accounting and have someone (a CFO maybe?) who can actually interpret SaaSOptics data for you, just go for it.
FirstOfficer.io doesn’t cover all the areas that SaaSOptics does, but my goal is that any entrepreneur could use it. No MBA/CFO skills needed. So I visualize just a couple of key things that no SaaS owner can afford to miss, but I visualize them well.
So the Company B should be able to grow smoothly to $35,000 or so before the growth ceiling will start to slow it down.
Whereas for Company A it will take just several months before it will have to start struggling against the growth ceiling.
Let’s take another long-term point of view
One way to think about business value is to imagine what would happen if you wouldn’t get any new customers.
How many months would it take until churn would leak you dry and all customers would disappear? What’s the value of your current customer base?
Company A average customer life-time: 7 months
Company B average customer life-time: 20 months
Normally you’d need to convert this to MRR. But these companies have same MRR, so we can skip that. Now we know that Company B is currently more valuable, more fail-safe and more stable.
The big tug of war
There are two factors here – the short term MRR and long term business value. And whether you do it actively or not, you are always optimizing between those two.
Just like you did when you decided to become an entrepreneur. You decided to invest your hours to your product, instead of selling them for immediate benefit. You did that because you knew that you’d eventually get way more money and freedom and other nice non-money things.
Now when your business is running you’ll need that investment mindset more than ever.
And this is where collecting the metrics and actually using them to do little calculations like we did here can really pay off. Otherwise it will be super hard to guess whether your main focus should be on acquisition or on retention.
Stop missing upcoming articles – drop your email in the box below:
When SaaS businesses grow they move though different phases. Sometimes it can be a painful experience.
You can ease up the growing pains if you understand what’s going on. So lets take a look what happens behind the scenes.
The SaaS ecosystem
You have a certain audience. These are the people who in general could buy your app, but you can only reach a part of them, the leads.
Part of the people you reach will become your customers and create new MRR (monthly recurring revenue).
Those people will move into the next category, and become your existing customers. They’ll give you recurring revenue every month.
Each month a part of your existing customers leaves, and take a part of the recurring revenue with them, the lost MRR.
The proportional relationship between new customers and leads is called conversion rate. For simplicity’s sake, let’s imagine that each customer pays $1, so I can refer to conversion rate and mean the relationship between new MRR and leads.
Likewise, the ratio of lost MRR to MRR is called churn rate. But it too could be measured in customers as well.
Rules of the game
You can scale any of the colored boxes up or down. But while you do that, the green new MRR box scales along with the leads box and the red lost MRR box scales along with the MRR box.
Running a marketing campaign to reach more people would scale up the yellow leads box, and the green new MRR would follow.
A/B tests to increase conversion rates would scale up just the new MRR box
Upgrading your existing customers to more expensive tier would scale up the MRR box, and red lost MRR would follow.
Making customers happier would scale down only the red lost MRR box
Now, let’s see a couple of examples. I’ll be using 25% conversion rate and 25% churn rate – just because that’s 1/4 so it’s easy to draw.
Realistic figures for a plateauing SaaS would be e.g. 2% conversions and 10% churn/mo. Yes, per month.
A baby SaaS ecosystem
At the start your existing customer base and MRR is small. But there are a lot of people out there in the leads box.
This is just a plain game of percentages, so it’s much much easier to make a visible impact by concentrating on the bigger mass of people. So you’ll focus on the left side leads and new MRR. And growth follows.
A teenage SaaS ecosystem, plateauing
When your customer base & MRR grows, the lost MRR grows along. When the lost MRR approaches the size of the new MRR, your business growth first slows down and then stops. For every new customer you win, one of your old customers leaves.
￼But as you’ve used to work on the left side, you may fail to notice that your existing customer base is now a considerable mass of people, and you keep on targeting your improvements to the leads and customer acquisition.
That’s what you’ve always done and it has worked.
What happens is that you do get growth, but as the left side feeds the right side, MRR grows and also scales up the lost MRR. Lost MRR catches up with your new MRR again.
So you work to hard to break the plateau, but the growth is withering, like you’d be pushing against a headwind. Your MRR may look something like this:
So what’s the secret?
You may have guessed it already.
You will need move your main focus from potential customers and acquisition to your existing customers and retention. But at the same time, you can’t decrease your acquisition efforts or the churn will eat you up.
You’ll need to adjust to the new environment. What boosted your business growth earlier stops working. Your old bag of tricks doesn’t help.
Edit: After analyzing a lot of SaaS metrics for FirstOfficer.io customers, I know that it’s possible to have this kind of plateau before your customer base is large. That happens when you don’t have enough traffic or conversions and in those cases the focus must remain in acquisition. Always analyze the metrics in their context!
Of course it depends on your business where the sweet growth spots are. The actions that affect the right side will often need implementation on the acquisition and marketing side, because the leads side feeds the MRR. But it’s a completely different action to change your marketing to acquire more of a certain type of customers than just to change it to acquire more customers in general.
Worse, when you’ve worked in a certain environment for years, you may totally miss several pitfalls around this change.
1. You run into the plateau in full growth mode
Businesses invest in their growth. And when you are bootstrapping, you often forego profit and just invest it all to growth. Or most of it.
The idea is that you’ll grow you business and THEN you collect the profit.
But here’s the bad news. When you are in plateau, you must keep on investing that money, or your SaaS starts to shrink. Your new MRR isn’t growing your business, it’s just replacing the lost MRR.
Yes, lots of people are coming in from the door. But your business is like a revolving door. As one person comes in, another steps out.
There’s a Finnish saying: “Everything is possible, except skiing through a revolving door”.
Unfortunately your customers aren’t on skis and don’t get stuck. Bye bye profits.
2. “We’ll just fix this quickly like we always do”
The actions that target leads or new MRR have often immediate and radical impact. Not so with the actions that target your existing customers.
Just think about it. A/B testing, pricing changes and conversion optimization – immediate results.
But if you want to cross-sell or up-sell to your existing customers, you’ll have to first build the thing that you want to sell. If you want to lower the churn by changing your customer base composition to better match your ideal customer, those new customers come in tiny drops relative to your old customer base.
And if you aren’t prepared at all and have to start collecting data for profiling your customers from the scratch, it will take 6-12 months to collect a comprehensive data set. Before you have it, it’s harder to measure your progress. You’ll be pretty much throwing spaghetti on the wall and seeing if it sticks.
3. You keep looking for tips and tricks from wrong places
You may not have enjoyed reading David Skok’s or Lincoln Murphy’s articles because it seems like they’d be living in a different world with different rules, like: “over 10% churn means you don’t have a product-market-fit”.
And this far those rules haven’t matched your reality. You’ve maybe even gone: “He he, whatever, MY churn is 18% and MY business is growing – you bullshitters!”
So you rather read the blog posts of fellow bootstrappers. Those are the people that you relate to and whose advice has worked so well this far.
But David and Lincoln are working with post-plateau SaaS, while most of the blogging bootstrappers are running pre-plateau SaaSes (or in-plateau).
Don’t get me wrong, all of them give great advice – they just focus on businesses in different stages. And when your SaaS plateaus, you are moving towards the world and the rules David and Lincoln play with.
So you may want to dig up those articles on negative churn, customer success, deadly customer acquisition costs and finding your ideal customers that didn’t make sense before.
But I especially enjoy reading Jason Cohen. Even though his company is big, he has the talent of explaining things in a simple way. For example following up SSEBITDA now might save your ass later.
Prevention is better than cure
This type of plateau doesn’t have to come as a surprise. It gives clear warning signs to your metrics months before it happens.
I try to catch SaaS businesses to FirstOfficer.iobefore their growth plateaus. It has a view that shows changes to growth in MRR component level, so it’s easier to notice the approaching plateaus.
If you start working to clear up the issues well in advance and take care of your profitability, life will be so much easier.
Of course, depending on your numbers, it may be that your SaaS is just reaching it’s normal maximum size and you may not want to grow it further. But it’s still nice to know where you stand.
I was talking with one of the FirstOfficer.io testers, a smart and successful entrepreneur, when he said, “People are just guessing, aren’t they?”
“They want to look good, so they say that they had data, but I think they just guessed. And just look at what happened to Ron Johnson when he went from Apple to J.C. Penney. Even the big names fail.”
That point of view was totally new to me.
Finance and metrics are as much art as they are science. And when you understand data-based decision-making, you’ll know it always requires also vision and guts. But if you aren’t using data to back up your decisions now, you can have hard time imagining what actually can be done with it.
Now knowing that you don’t know is a risk
When I was a teen, my dad had me to do the company books. But a couple of years later when I went to study accounting I totally failed my first exam. Why? Because I thought I already knew all about accounting. My practical knowledge and confidence blinded me from seeing that the topic had much more depth than I could have imagined.
The biggest risks in your business comes from the things you don’t know that you don’t know.
Trying to use data to run your business if you don’t know the basics is risky. You’ll be confidently making bad decisions.
So let’s go through a couple of basic concepts. This is not specific to SaaS metrics, but general finances/metrics stuff.
It’s not a choice between guessing and knowing
When you use data to help you decide things, it does not mean that you would gain 100% knowledge. That’s just impossible. You’ll increase the probability that your guesses will be right. And you’ll manage the risks better. But there’s always some guesswork included.
You’ll go from a total guess to knowing that if things X,Y and Z will stay the same, you’ll reach your target. But you may also know that if thing X changes, it will be critical and you will fail. And there’s no way to ensure that things wouldn’t change.
As you can see, even when you have the data you cannot know the final outcome of your decision. But at that point, it’s hardly a guess either. It’s something in between. An educated guess, maybe?
You can’t separate your guts and the data
How you interpret the data depends on your skills, past experience and even your personality. You can give the same metrics dashboard to two different people and they see different things. It’s like everyone would look at the data through tinted lenses.
And you tend to ignore what you can’t understand.
For example, when I didn’t know how to read balance sheets, I thought it was just a trivial report that the law required us to print out every month. I thought that only the P&L sheet mattered. But it was just me who couldn’t even imagine all its purposes.
Numbers are not facts
You may think that numbers are facts. But when we are talking about finances and metrics, they aren’t. They are just a rough way to try to model the reality.
You’d think that profit from your P&L sheet was a hard number. Surely what you earn is a fact. But it isn’t. It’s your accountant’s best effort to match what really happens.
And the same goes for the SaaS metrics, but with somewhat smaller impact. Your last month MRR may contain people that end up asking full refund in the future. And how did you categorize that person who bought 3 subscriptions and then cancelled 2? Were those downgrades? Lost subscriptions? Or a clear mistake that should not be visible in the figures at all?
The only number-related thing that’s really a fact is the number of dollars in your bank account. Cash is the king.
If this sounds too strange, there’s an excellent book called Financial Intelligence that teaches basic finances to entrepreneurs. It’s targeted to larger non-SaaS companies with accrual accounting, but it’s still a great read, especially if you are also interested in investing.
Past performance does not guarantee future results
Some of the SaaS metrics are in fact future-looking projections, like customer lifetime value (CLTV). You are taking figures from your historical data and trying to estimate what will happen in the future.
It’s impossible to produce fully reliable CLTV figures.
For starters, there is a handful of different formulas to use, each one optimized for certain conditions. Churn rate, a major component in calculating the CLTV has lots of room for interpretation and inaccuracy too.
Secondly, even if the figure would be somewhat right, the behavior of your past customers can never predict how your existing or future customers will behave.
But having figures that are not perfect is OK – you just need to know that.
When you know which numbers are trustworthy and which are not, you can team up reliable figures together with the weak ones. That way you can come up with a prediction that you can actually work with.
Following up the numbers vs. proactively using them
I hope this post has given you a glimpse to the concepts behind data-based decision-making.
It has been educating for me too. I can now see what a huge jump it can be to go from following up your metrics to actually effectively using them in decision-making. And that information will definitely help me in FirstOfficer.io development – I don’t want it to be just another metrics dashboard. I want it to produce real results.
If you liked this and don’t want to miss up the upcoming Happy Bootstrapper articles, drop your name below:
If you want to get rid of the every-month fiddles with metrics spreadsheets, here’s the first step – automatically calculating monthly recurring revenue from the payment gateway entries.
MRR is the number 1 metric for any SaaS business, and many other metrics base to it.
There are quite a lot of steps, I’m sorry for the long post.
Real or theoretical plan price?
MRR is often calculated using the theoretical plan price. So if Sally has a $5 discount on her $25 per month plan, her MRR would be $25.
But when you have the real entries from Stripe API, you’ll probably want to calculate MRR with real amounts instead. So if Sally pays $20 per month, $20 is the recurring revenue from Sally, not $25. This way the MRR will show the actual amount you’d really earn next month if none of the current customers would churn, upgrade or downgrade.
If you also calculate the discounts & refunds separately, you’ll see the MRR you lost due to them.
Don’t make this common mistake!
Monthly amount billed from subscriptions is not your MRR. Not even if you’d have just monthly subscriptions.
Due to changes in refunds, discounts and delays in payments the MRR figures calculated from real paid amounts don’t normally match month-to-month.
Calculating SaaS metrics isn’t accounting. This said, let’s get started.
The amount field will contain the amount paid. We’ll not include Stripe fees in the MRR. Leaving them out makes it easier to do profitability calculations, but it’s up to you if you want to include them.
BalanceTransactions do not include everything as Stripe keeps internal customer balance, but let’s handle that later.
Next, we’ll need to find the plan that should carry the MRR. Even though we are calculating just the total MRR here, we’ll need to know the plan length so that we can normalize the MRR for annual subscriptions.
How to find the plan
The BalanceTransaction entry has a source field that contains the Charge id. The Charge has an invoice field that contains the Invoice id. The Invoice contains invoice lines, which may contain a “subscription” type entry. And those subscription type line items contain the plan information.
The invoices also contain “invoiceitem” type of entries, which contain anything else that’s invoiced on top of the monthly charge. These invoice item entries should not be included in the MRR, unless they really are recurring.
For example, if you are charging extra for customer’s disk space usage, and he can’t easily cut it down, include invoice items. But if you charge extra for every report that he prints out, and he prints them whenever he wants, don’t include invoice items to MRR.
How to find the amount
The invoice lines contain the theoretical plan price. Invoice’s total field contains the discounted total that we want to use. Unfortunately, you don’t get discounted line amounts, but the discounts can’t be targeted to single lines either, so for now you can just split the total among the invoice lines and include the lines belonging to MRR.
If you don’t charge invoiceitems, the invoice will contain just the subscription line item and you can pick up the total as it is.
Normalizing the MRR
If the invoice you are handling is to a monthly plan, just include the MRR found in the last step.
But if the invoice is to an annual plan, we’ll need to split it correctly. Subscription type invoice lines contain period object that hold the period start and period end for this subscription. You’ll need to split the total to the subscription period. For example, an annual subscription starting at 15th of January 2014 would be split to 12 parts, targeted to Jan-Dec. So no MRR to Jan 2015 for that plan!
You can assign the amounts to the right months right away. There are extra benefits from doing that. Or, you can just include 1/12th of the total and take that subscription item into your calculations during the next 11 months too.
Accounting the MRR to the right month
You should use the subscription’s period start and period end to assign the MRR to the correct months, even when you are handling monthly subscriptions. If user has problems with their card – and they often do – the invoice belonging to one month may get paid in the next.
Handling refunds and special cases
At this point we have the basics down, but we still need to handle special cases and refunds.
Handling refunds is done by reversing the process we did earlier.
BalanceTransactions of type “refund” have negative amount and a link to the Charge that was refunded. You’ll calculate the part of the refund that needs to be removed from current month’s MRR using the same rules that we used for calculating the MRR itself.
The refund needs to be normalized too. A refund given today to a monthly subscription may well affect previous month’s MRR. A refund given today to an annual subscription may affect up to 11 months in future or history. But it’s up to you if you go back in time and fix previous months’ MRR.
For example, if customer bought an annual subscription 6 months ago, and you’ll have to refund her the whole amount now, it’s up to you if you want to fix up the MRR entries from 6 months ago, or if you’ll just fix the entries for the current and future months.
You’ll also want to check from Customer’s Subscription data if the customer was lost, especially if whole MRR got refunded. Otherwise your customer counts will be wrong and you’ll have ghost customers with $0 MRR.
Including payments from customer balance
Stripe keeps an internal balance for each customer, and when they unsubscribe from an annual plan and move to a monthly plan, the difference does not get refunded to them, it goes to their balance. And as long as they have balance, you don’t get any BalanceTransactions when they pay something. You also don’t recognize that they have changed plans until their balance runs out.
To get your MRR right, you’ll need to include Charges paid from balance.
Here’s how you recognize Charge paid from balance:
The Charge paid from balance is marked as paid, but no amount was due.
Prorations happen when customers change plans, and you haven’t disabled the prorate option in Stripe. Stripe creates an imaginary refund to the customer for the part of the subscription that the customer didn’t use, and place a new subscription to the new plan.
There are 2 types of prorations, ones that happen between plans having similar billing period and ones that happen when billing period changes.
Unfortunately both types mess up MRR calculations.
Handling Prorations when billing period changes
This type of proration, if not handled, will cause you to show MRR to wrong plans, but does not affect the total MRR.
As we are calculating just the total MRR here, I will not go into details, but the trick is to properly assign the refunds to the right plans, and create a correction entry so that the amount lost/refunded matches the whole subscription amount that the customer actually paid.
You’ll have extra trouble finding out the plan that was refunded, as the plan id is not returned in the Stripe proration invoiceitem.
Handling Prorations when billing period does not change
This type of prorations, if not handled, may cause you to show extra MRR for a single customer. The proration collects up changes that may have happened in previous month, and then charges them in the next.
For these prorations, you’ll need to check if the MRR to the new plan belongs to the previous month, and if it does, you’ll need to ignore it.
For example, you had a customer who subscribed to $9.99/mo plan in 27th of Nov. In 2nd of Dec, he decided to move to $14.99 plan.
Stripe will give you this proration at 27th of Dec: $8.47 was refunded to the old plan, $12.70 was charged from the old plan’s remaining period (using new price), and $14.99 gets charged from the new subscription.
In this case, extra $12.70 would be added to second plan’s MRR and extra $4.23 would be added to total MRR, if you handled this wrong. The right MRR for Dec would be $14.99.
If, instead, this customer would have made the change in 28th of Nov, you would have needed a different handling. In that case you could make the move right there and then and account $14.99 as November’s MRR for that customer.
Should you really do all this?
Now that you know how to calculate MRR from Stripe entries, should you do it?
Well, the question to ask is: “Will doing this give me the best return of investment to the time spent?”
If you are just starting out, it may be better to just keep count on customers and calculate MRR with plan prices. Or to implement just the part handling charges and refunds and to keep count on the customers who change plans. Those are the ones messing up the figures, and if you don’t have plenty of them, the difference isn’t that big.
What I went through here is just calculating the total MRR, and getting the MRR by plan right is a lot more work still.
To be frank, I probably would not have done all this if I wouldn’t be implementing an analytics app. If it would be ready I’d say: “Don’t do this yourself, buy First Officer!”
I was happily boarding early testers to my upcoming Stripe analytics app, when the logs got filled with “Error R14 (Memory quota exceeded)” warnings. I’m new to both Ruby and Rails, so I thought “Gee, time to download a profiler”.
There are also some older profilers (MemoryProf) that just don’t work with Ruby 2.0 and Rails 4.0. Ruby-Prof, which does work, does not give you information about the memory usage. It writes html files, so it’s more suitable to be used in development anyway.
So, I cried a little. Then I rolled up my sleeves and frankenstained a somewhat working solution from the pieces that I had.
1. Confirming the leak/bloat
Heroku can write summaries about memory usage.
Just run heroku labs:enable log-runtime-metrics for your app.
Then, run the problematic worker again and follow the logs. Now you can see the memory usage statistics in between your own log entries and you can easily estimate if the memory behaves like expected.
In my case, the problem worker was fetching all customer’s data from Stripe. It had several separate sections, and in between them, the garbage collection should have picked up the trash. As they say Ruby MRI doesn’t really give you memory back if it gets reserved, I was expecting an upwards trend with plateaus every now and then when garbage gets collected. What I got was a constant upwards trend. A memory leak. Or a bloat.
You’ll also want to set up the New Relic monitoring for your app.
If you put GC::Profiler.enable to config/initializers/new_relic.rb, you can see your memory usage in New Relic.
New Relic also has a thread profiler that can see your Sidekiq worker. In my case, these tools were nice for checking out the overall situation, but didn’t give me enough details. Maybe the tools in the paid version are different.
The top part shows the memory usage in KB. The next part shows the garbage collector stats. Here are instructions for reading them. And the bottom shows the object instances in memory.
In development env it will run also RubyProf, and it will write the results files in your app’s log dir. In Heroku it will print the results to the log, and skip RubyProf.
Start it with MemTool.start(“some info, like class name”) and stop it with MemTool.stop.
I wish I would have been able to find out who reserved the objects, but I’m still newbie enough that patching Ruby runtime sounds too scary. If you have any advice/experiences on that, let’s hear it in the comments.
You’ll also want to run Oink. It tells you if you let ActiveRecord queries eat up the memory. Remember to use the -r option, or you won’t see the ActiveRecord stats.
Unfortunately Oink is meant for profiling the front-end, so you’ll also need to create a new controller request that calls the code under test. That means you’ll have to do this test in development env, as Heroku will kill your non-responding requests after 30 seconds.
Edit: After a couple of months I found a better way to check for ActiveRecord bloat and that finally helped me to fix the memory leak/bloat.
I installed the Rack-Mini-Profiler, loaded my development environment with data and created a temporary controller & view that only run the background task. Then I was able to use the Mini-Profiler’s great UI to check the actual SQL queries by the time they took. When you load a lot of things to memory, it takes time. So I just checked out the queries that took the most time and quickly found the bloat.
In my case, it was a total newbie mistake. I was calling blank? on large scoped searches.
3. Profile the job without Sidekiq
You can run your background jobs using rake in Heroku. Just call your worker’s #perform from a rake task. This way you get to take Sidekiq out from the equation and you’ll find out if the leak happens also when you are running outside of it.
4. Fix things
That’s the phase I am. I was able to find several problem spots from my code, which I’m fixing now.
After reading couple of articles by Mike Perham, the creator of Sidekiq, I don’t think I can fix my workers to be bloat-free, but I’m targeting to getting them into a condition where once-a-day restart with Monit would be enough.
I’m also trying to reduce the base memory usage, in case I need to switch to Resque. Resque forks a new process for every job, loading the whole env each time – that’s why it in general needs more resources than Sidekiq. But that would also mean that every job has a clean plate.
I’ll write a follow-up when I’m done, and if you don’t want to miss it, you can drop your email here:
And then you can interpret the metrics wrong. Today I’ll present you the 3 simple interpretation mistakes that you can avoid just by knowing about them.
1. Compare Conversion Rate to Churn Rate
Business growth stops when more revenue is lost with churned customers than you get from new customers. That’s why people have the inclination to put the churn rate and the conversion rate side by side and compare them.
But you can only compare rates when they are calculated from the same baseline, which isn’t the case here.
Conversion rate is calculated using potential customers as its baseline. Churn is calculated using existing customers who are able to churn as its baseline. These two rates are not comparable with each other.
Growth rate can’t be compared to churn rate either, for the same reason, but the mistake is smaller there.
One way to find out if you are approaching a plateau is to compare the actual MRR components. When the Lost & Downgrade MRR gets as big as New & Upgrade MRR, growth has stopped.
The screenshot is from our upcoming analytics tool.
It doesn’t matter how much you charge – it only matters how much of it you get to keep after you have paid the bills and the taxes.
This shouldn’t be applied only to revenue, but to all the key metrics using revenue as their base.
Customer Life-Time Value (CLTV) should be calculated from profit, but nowadays it’s often calculated from revenue. You don’t want to use 1/3 of the revenue from a single customer to his acquisition. But you may want to use 1/3 of the profit to the acquisition, if you can recover that money fast enough.
If you are using CLTV and you didn’t calculate it yourself, find out if it’s calculated from revenue or from profit. You can then multiply the CLTV (revenue) by your gross profit margin, and get the ‘real CLTV’.
3. Confuse MRR with Cash Inflow (or Sales, or Bookings or Revenue)