Part 1 of 3: How to Determine the ROI of Design

When you’re trying to figure out the return on investment (ROI) of your design projects, you’re going to hear a lot if input. It’s not possible. The numbers aren’t reliable. Those stats have no place in a serious boardroom.

We hear these responses and more all the time. But the truth is it is possible to determine your ROI of design. It just takes careful planning and execution.

Download the full, free white paper

Continue measuring your usual digital marketing metrics

Although your usual digital marketing metrics are subject to influences both internal and external, there is overlap between these stats and user experience stats, especially when viewed in the context of behavior patterns.

Common digital metrics like funnel abandonment rate, bounce rate, session duration, and of course conversions, all contribute to a complete picture of design’s effectiveness. If your team has recently redesigned the pages in your ecommerce checkout funnel, create a filter in your analytics software that isolates the traffic in that funnel. Make a note of when the new designs were implemented, and use these numbers as your baseline for overall digital health.

It’s important to consider these basic metrics in the long term as well. This is most effectively done through examining each of your main audience groups separately. Has the purchasing frequency or average purchase amount in your loyal customers segment increased? Has the average time-to-first-purchase decreased for customers at the top of the funnel? This is data that can only be reliably collected over time, so plan your overall reporting plan accordingly. And the insights can be profound - 50% of design-led companies report higher customer loyalty 3 as a direct result of their design efforts.

Behavioral data can provide extra context

Beyond the basic metrics, you need to keep a close eye on the behavior flows your users are taking before and after the design implementation. Maybe the changes resulted in a decrease in abandonments at the pricing page, but an increase in the shipping and returns page in the funnel. Assuming the new pages were the only substantial changes made, any shifts in user behavior can be at least partially attributed to the new designs.

Basic digital marketing statistics and bottom funnel business goals like conversions are subject to too many influences. But how users navigate your site and the different journeys they intuitively take through it, when using historical data as a baseline, can provide strong correlations to measure your UI design work by.

Specifically in the context of user experience design, few tools can be more helpful in behavioral analysis than heat mapping and visitor recording software. Watching how users interact with your site in real-time provides invaluable data for designers. Beyond page-to-page behaviors, witnessing the moment of engagement and the placement of the click down to the pixel can sharpen your tactics to near perfection.

A/B Testing: The Product Cage Match

It’s time tested and proven effective. If you want to measure the impact of a new change, put it right up alongside the old. In the case of design, this is in reference to A/B testing.

If you re-designed your website, A/B test the new pages against the old. The new may look obviously better to you, but the old (or baseline) variation may hold value for users in terms of intuitive (or familiar) layout, ease of navigation, better calls to action, and myriad other factors. Using heatmap software can help pinpoint where the old design excelled, and these elements can be implemented in the new design.

A/B testing is most useful for determining the ROI of major design changes. When a design system undergoes a total overhaul, there are essentially two variations to test against each other: the old and the new. But what about more subtle design updates, or identifying design’s ROI down to the component level?

Get Granular with Multivariate Testing

A/B testing can be acceptable when all you need is a high level look at changes and their potential effects. If you want to get more granular with your measurement and see which design elements are having which specific effects, you need multivariate testing. With this methodology you are able to assign values generated by individual components within a design system.

If you have three components, and each component has two variations that you want to test, then your total number of possible variations is eight (2 x 2 x 2). With multivariate testing, you create all 8 of the possible variations, and test them all. This will enable you to isolate the impact of a single component, rather than the effects of all components working together like you would with A/B testing.

If one variation produces a statistically significant increase in sales than the other seven variations, you have specific ROI data linked to specific design components. The multivariate testing method can be used to demonstrate the value created by more subtle design changes or updates.

The Digital Surgeons ROI of Design report goes into more detail on the steps and tools for effective complex measurements.

Bringing it Together with Customer Feedback

Customer feedback surveys are a longstanding pillar of design measurement. Taken on their own, however, they are not considered a reliable measurement tool because survey data is qualitative data.

This can be partially mitigated through the common Likert scale, which is when surveys ask question based on degrees.

How likely are you to use our product again?

1 (Very likely)   2 (I don’t know) 3 (Not very likely)

Allowing for qualitative variations while assigning numeric values is a good start, but this data still can’t be considered truly quantitative on its own. However, when this survey data is combined with the above measurement approaches, you can start to reveal trends, fill gaps in your insights, and craft a total narrative for your reporting.

Let’s say your basic metrics are showing increased conversions, your behavior flows are showing higher engagement and lower abandonments in the funnel, and a multivariate test is showing that the bulk of the impact can be attributed to two components. In your customer satisfaction surveys, you can find trends in responses that support these findings, lending even more credence to your design ROI calculations and assertions.