My last post showed why traditional KPI-based reporting – in both digital analytics and people-measurement – sucks. The problem isn’t inherent in the idea of KPIs. Finding a small set of important metrics makes pretty good sense. Unfortunately, when you deliver those metrics across all visits – whether to a website or a store – they become unusably noisy.

 

Across unsegmented populations, changes to the KPI value will nearly always reflect shifts in the population of visitors, not changes in performance. Worse, there’s no way to unpack which factors are driving the change or what the change means. So instead of simplifying the consumption of data and leading users to the important points, KPI reports nearly always mislead report consumers; they encourage them to draw specific conclusions from the data that are almost never correct.

 

So how does segmentation help?

 

Consider a retail store focused on a simple funnel: cart size (total purchase $ / purchases), conversion rate (purchases / visits), engagement rate (avg. dwell time) and traffic (visits). Those are four straight-forward and obviously meaningful KPIs for overall store performance. But without segmentation, they are dangerous and potentially misleading. That’s true both for comparing store performance over time and -even more so – for comparing one location to another.

 

The problem is all about intention and customer types. At most stores, somewhere between 5-20% of store visits are not sales focused and if groups are factored in, the number can be closer to 40%. At mall stores with outside access, more than half of all visits may not be potential sales conversions. A visit focused on product return, customer support, mall pass-through, or store pickup is much less likely to drive an incremental sale than a typical shopping visit. That makes KPIs like conversion rate, engagement rate and traffic vulnerable to distortion. Changes in the mix of visit types will drive changes in the KPIs. And differences in the mix of customer visit types will make comparison of individual locations almost completely meaningless.

 

Similarly, metrics like cart size are often a function of the starting intent of shoppers. How do you separate out the mix of shoppers you’re getting from the performance of the store? It’s important to do that because changes in the cart size metric may indicate changes to store effectiveness, seasonal shifts in product focus, or changes in the quality and type of shoppers coming to the store. And, of course, they may also reflect competing trends in each of those that cancel out whatever signal might exist in the underlying data.

 

I’ll say it again for emphasis – if your business has more than one type of visitor (and every business does), KPIs are uninterpretable without segmentation.

 

But can behavioral segmentation help classify the intent of shoppers? Isn’t that something only survey research could do?

 

Fortunately, behavioral data is surprisingly telling when it comes to visit intent. At a typical retail store, behavioral analytics can classify customer support and product return visits (navigate from entry to customer support/returns/check-out), pick-up in store visits (first stop at pickup location – even parking in a pickup location), mall pass-through visits (no dwells from store to mall entry), groups, shoppers with cart (that’s intent to buy), and initial navigation path (which can classify shoppers by initial product intent).

 

With this kind of segmentation, you can clean-out visits that have little or no shopping intention AND you can segment out degrees of intent and potential value to better understand store and marketing performance.

With this kind of visit segmentation, you can compare locations with far less noise, and you can draw comparisons down to specific shopping segments. You can monitor changes in the distribution of shopper types to analyze your broader marketing. And you can measure performance within shopping segments to evaluate actual store improvement.

 

Almost every retail location has customer segments like these and most companies have additional visit segmentations unique to their business and customers. Consider, for example, how restroom use impacts pump to store percentage, store conversion rate, and $ per store visit for a gas station/convenience store like the one highlighted in the video I posted last week.

 

Many people will enter the store just to use the restroom. You might sell them something, but the odds are against it. So, if you have a shift in the percentage of people getting gas who need to use the restroom, all of your KPIs are going to change.

 

You may think that’s unlikely, but I’m here to tell you that it isn’t.

 

Suppose, for example, that a station is near a freeway that sees a dramatic increase in travel with summer tourism. Through-drivers are far more likely to use restrooms than local drivers. So as seasonal shifts in usage play out, your KPIs will all change. If management reads those changes as reflective of actual location performance, they’ll be misreading the data and probably making bad decisions.

 

Or suppose that one location has outdoor restrooms, and another has indoor restrooms. The location with indoor restrooms will look like it has much better pump to store percentages but much worse conversion rates and lower $ per store visit.

 

The noise from different visit types make unsegmented KPIs very difficult to interpret. But if you are segmenting every visit type for a store, instead of reporting location-wide behavioral KPIs, you’re reporting on the segment mix, how it is shifting, and KPIs for each segment.

 

When a report consumer looks at a report, they’ll see that Store X has Restroom Visits (Visits that start with a restroom visit) and Store Y doesn’t. No confusion there. What’s more, their pump to store and conversion metrics will no longer be confused by all those restroom visits. Every KPI for every other segment is instantly cleaner. Finally, they’ll be able to see KPIs for visits that START with a restroom visit. Because you might be able to sell those people something and you want to think carefully about restroom placement and what kind of impulse purchases might be optimized. If you haven’t segmented out those visits, the chance of you EVER detecting a change in that population’s behavior is essentially nil.

 

Segmentation is the difference between actionable/intelligible data and useless/misleading data.

 

You may think restroom users are highly specific to this business. That’s true. But EVERY business I’ve ever worked with has unique customer segments. If you’re just looking at top-level behavioral KPIs, nine times out of ten, shifts in those KPIs or cross-location comparisons of those KPIs reflect changes in the underlying population of visit types not in location performance. Understanding the difference between those two is something your top-level KPI report not only doesn’t illuminate, but flatly misinterprets.

 

Honestly, our people-measurement platform hasn’t been great at supporting this kind of visitor segmentation. It’s hard to deliver and there hasn’t been much demand. We live in a space where people are still getting used to having the most basic data, but that’s changing as our clients grow more sophisticated and we’re greatly expanding our capabilities to support this kind of segmentation. And, of course, it’s been many years since digital analytics consumers were struggling to understand the basics. Yet, I still see plenty of reports that feature un-segmented KPIs and plenty of experts who tout KPIs as the best way to deliver quick business intelligence.