Forget ROI, Use MoAR For Product Portfolio Prioritization

Summary: If you measure your product’s success in more than just monetary value, you should replace ROI with MoAR for effective product portfolio prioritization.

Every department has a key metric. For sales, it’s revenue. For engineering, velocity. Customer success has retention, Marketing has MQL. But what about Product? What metric measures Product’s effectiveness?

Product leaders don’t have a primary KPI - the old school ROI does not work for Product if you measure your product’s success in more than just monetary value.

You need to replace ROI with MoAR to measure the effectiveness of product.

Why? Because Metrics on Available Resources (MoAR) is a more immediate measure than Return on Investment (ROI) when evaluating opportunities that maximize product and business outcomes. 

MoAR is a key element of the new generations of PPM - the Responsive Portfolio Product Management.

Years ago, I led strategic planning for our product and technology organization. We needed to evaluate all the potential opportunities that may help the company to achieve its growth, product innovation and financial objectives.

These opportunities range from improving engagement, launching into a new market, reducing transaction costs, and taking risky bets on emerging trends.

How could we compare these diverse types of opportunities?

We started using ROI, like most companies do, but quickly we ran into many limitations.

What’s ROI?

ROI, as Return on Investment, is a ratio between net profit (over a period) and cost of investment. ROI is a performance measure often used to evaluate or compare the efficiency of a number of different investments. (wikipedia)

ROI works well in the traditional industries or functions where benefit can be effectively measured in $ and scarce resources are also $.

But ROI does not work well for companies or functions that need to measure outcome in factors beyond $, or have other forms of scarce resources than generic money.

On measuring benefits:

In the world of manufactured products, measuring the financial impact of a new product is relatively clear. A new designed cup is built and sold. We can measure how much revenue this new product generates in 1 year and figure out the total costs to calculate the 1 year ROI of this new cup.  

A software product consists of many features. A proposed new feature may have a relative contribution to a business metric, which may be used to infer a monetary benefit. But it’s nearly impossible to directly measure and attribute the financial outcome of individual product features. 

Let’s take a look at an example. Based on past experience, we know that increasing sign up conversion by 5% will result in $100k in quarterly profit if everything else remains unchanged. Assume Feature A aims to increase sign up by 2%. Based on the information above, we can infer that 2% improvement would result in $40k profit. At the end of the quarter, we can measure the change of profit, but it is impossible to attribute the profit change to a given feature. Therefore using the financial impact of a product feature does not provide feedback on the benefit of a feature. On the other hand, measuring the sign up conversion ratio over time can be easily achieved. 

A monetary based benefit measure also has a negative bias against innovation and risk-taking. The same amount of investment on the cash cow mature business always generates a much higher $ return, than investing in nascent products. In the podcast of “What Really Happened When eBay Went To China”, Alan Tien, the former GM who launched PayPal China, highlighted how difficult it was to justify the R&D investment in improving product features in a smaller market. (relevant discussion at4:30 at the full podcast here).

Measuring benefit in financial term does not differentiate the type of investment being made. Measuring benefit in metrics provide explicit indication on the type of the investment. 

On measuring cost:

In technology companies Money often is not the most scarce resource. The more scarce resources are human capital. For example, currently there are more than half a million unfilled engineering job openings. $1 million does not equal to the 4 senior Python engineers the company needs now to build product and achieve its growth metrics.

So, what’s MoAR?

The Metrics over Available Resources (“MoAR”) is a ratio of relevant contribution of a product feature towards a metric used to measure a business outcome, over the amount of resources needed to achieve this contribution.

MoAR enables a more direct and holistic measure for portfolio roadmapping.

How does MoAR fit product planning?

Start with using appropriate metrics to measure benefits towards a desired business outcome.

Metrics, e.g. user growth, referral rate, retention, NPS, can be used to measure product benefits. When defined well, they are easily measurable. They are often leading indicators to $ based returns over time. Companies adopt OKR or V2MOM framework can easily connect outcome targets with product portfolio roadmapping. This enables outcome driven portfolio roadmaps.

Available Resources is a better indicator of scarce resource. Examples are the Scala Engineers or the UX designer who designed the last app.

During portfolio roadmap planning, we compare features that require the same resources using MoAR to prioritize.

We also add up the contribution of all planned features towards the goal. The total contribution should add up to 100% or higher to ensure the goal is achievable based on the current plan, with the resources we have. This exercise exposes any known outcome gap and/or resource gap early on. 

For example, assume a key business goal for the next quarter is to increase conversion by 10%. 

Three features are coming to the team and there are only 16 weeks of front end resources available.

• Feature A is expected to contribute to 40% of this goal and requires eight weeks of front end resources. The MoAR for Feature A is 40/8 = 5
• Feature B may add to 20% of this goal and requires eight weeks of front end resources. The MoAR for Feature B is 20/8 = 2.5
• Feature C contributes to a cost-saving metric, but not towards the conversion goal, and requires four weeks of front end resources. The MoAR for Feature C is 0. 

First, we notice the shortage of front end resources required to support these three features. We will de-prioritize Feature C as it’s MoAR is 0.

But, this plan is not ready to go yet because you can see there is an outcome gap – Feature A and B will contribute to only 60% of the target metric. 

The team needs to either reduce the target outcome by 40% and get stakeholder buy-in or need to start over to design a different feature set and/or ad other levers to achieve 100% of the target.

Identifying these gaps during the planning phase gives the team more time to address them before it’s too late.

The Metrics over Available Resources ("MoAR") method allows a technology company to readily incorporate a direct and holistic measure into their roadmap planning/ dynamic alignment process. This enables better product decisions and overall outcome.