The equation in retail today is simple. Evolve or die. But if analytics is one of the core tools to drive successful  evolution, we have a problem. From an analytics perspective, we’re used to a certain view of the store. We know how many shoppers we get (door counting) and we know what we sold. We know how many Associates we had. We (may) know what they sold. This isn’t dog food. If you had to pick a very small set of metrics to work with to optimize the store, most of these would belong. But we’re missing a lot, too. We’re missing almost any analytic detail around the customer journey in the store. That’s a particularly acute lack (as I noted in my last post) in a world where we’re increasingly focused on delivering (and measuring) better store experiences. In a transaction-focused world, transactions are the key measures. In an experience world? Not so much. So journey measurement is a critical component of today’s store optimization. And there’s the problem. Because the in-store measurement systems we have available are tragically limited. DM1, our new platform, is designed to fix that problem.

People like to talk about analytics as if it just falls out of data. As if analysts can take any data set and any tool and somehow make a tasty concoction. It isn’t true. Analytics is hard work. A really great analyst can work wonders, but some data sets are too poor to use. Some tools lock away the data or munge it beyond recognition.  And remember, the most expensive part of analytics is the human component. Why arm those folks with tools that make their job slow and hard? Believe me, when it comes to getting value out of analytics, it’s hard enough with good tools and good data. You can kid yourself that it’s okay to get by with less. But at some point you’re just flushing your investment and your time away. In two previous posts, I called out a set of problems with the current generation of store customer measurement systems. Sure, every system has problems – no analytics tool is perfect. But some problems are much worse than others. And some problems cripple or severely limit our ability to use journey data to drive real improvement.

When it comes to store measurement tools, here are the killers: lack of segmentation, lack of store context, inappropriate analytics tools, inability to integrate Associate data and interactions, inability to integrate into the broader analytics ecosystem and an unwillingness to provide cleaned, event-level data that might let analysts get around these other issues.

Those are the problems we set out to solve when we built DM1.

Let’s start with Segmentation. Segmentation can sound like a fancy add-on. A nice to have. Important maybe, but not critical.

That isn’t right. Marketing analytics just is segmentation. There is no such thing as an average customer. And when it comes to customer journey’s, trying to average them makes them meaningless. One customer walks in the door, turns around and leaves. Another lingers for twenty minutes shopping intensively in two departments. Averaging the two? It means nothing.

Almost every analysis you’ll do, every question you’ll try to answer about store layout, store merchandising, promotion performance, or experience will require you to segment. To be able to look at the just the customers who DID THIS. Just the customers who experienced THAT.

Think about it. When you build a new experience, and want to know how it changed behavior you need to segment. When you change a greeting script or adjust a presentation and want to know if it improved store performance you need segmentation. When you change Associate interaction strategies and want to see how it’s impacting customer behavior you need segmentation. When you add a store event and want to see how it impacted key sections, you need segmentation. When you want to know what other stuff shoppers interested in a category cared about, you need segmentation. When you want to know how successful journeys differed from unsuccessful ones, you need segmentation. When you want to know what happens with people who do store pickup or returns, you need segmentation.

In other words, if you want to use customer journey tracking tools for tracking customer journeys, you need segmentation.

If your tool doesn’t provide segmentation and it doesn’t give the analyst access to the data outside it’s interface, you’re stuck. It doesn’t matter how brilliant you are. How clever. Or how skilled. You can’t manufacture segmentation.

Why don’t most tools deliver segmentation?

If it’s so important, why isn’t it there? Supporting segmentation is actually kind of hard. Most reporting systems work by aggregating the data. They add it up by various dimensions so that it can be collapsed into easily accessible chunks delivered up into reports. But when you add segmentation into the mix, you have to chunk every metric by every possible combination of segments. It’s messy and it often expands the data so much that reports take forever to run. That’s not good either.

We engineered DM1 differently. In DM1, all the data is stored in memory. What does that mean? You know how on your PC, when you save something to disk or first load it from the hard drive it takes a decent chunk of time? But once it’s loaded everything goes along just fine? That’s because memory is much faster than disk. So once your PowerPoint or spreadsheet is loaded into memory, things run much faster. With DM1, your entire data set is stored in-memory. Every record. Every journey. And because it’s in-memory, we can pass all your data for every query, really fast. But we didn’t stop there. When you run a query on DM1, that query is split up into lots of chunks (called threads) each of which process its own little range of data – usually a day or two. Then they combine all the answers together and deliver them back to you.

That means that not only does DM1 deliver reports almost instantaneously, it means we can run even pretty complex queries without pre-aggregating anything and without having to worry about the performance. Things like…segmentation.

Segmentation and DM1

In DM1, you can segment on quite a few different things. You can segment on where in the store the shopper spent time. You can segment on how much time they spent. You can segment on their total time in the store. You can segment on when they shopped (both by day of week and time of day). You can segment on whether they purchased or not. And even whether they interacted with an Associate.

If, for example, you want to understand potential cross-sells, you can apply a segment that selects only visitors who spent a significant amount of time shopping in a section or department. Actually, this undersells the capability because it’s in no way limited to any specific type of store area. You can segment on any store area down to the level of accuracy achieved by the collection architecture.

What’s more, DM1 keeps track of historical meta-data for every area of the store. Meaning that even if you changed, moved or re-sized an area of the store, DM1 still tracks and segments on it appropriately.

So if you want to see what else shoppers who looked at, for example, Jackets also considered, you can simply apply the segmentation. It will work correctly no matter how many times the area was re-defined. It will work even in store roll-ups with fundamentally different store types. And with the segment applied, you can view any DM1 visualization, chart or table. So you can look at where else Jacket Shoppers passed through, where they lingered, where they engaged more deeply, what else they were likely to buy, where they exited from, where they went first, where they spent the most time, etc. etc. You can even answer questions such as whether shoppers in Jackets were more or less likely to interact with Sales Associates in that section or another.

Want to see if Jacket shoppers are different on weekdays and weekends? If transactors are different from browsers? If having an Associate interaction significantly increases browse time? Well, DM1 let’s you stack segments. So you can choose any other filter type and apply it as well. I think the Day and Time part segmentation’s are particularly cool (and unusual). They let you seamlessly focus on morning shoppers or late afternoon, weekend shoppers or even just shoppers who come in over lunchtime. Sure, with door-counting you know your overall store volume. But with day and time-part segmentation you know volume, interest, consideration, and attribution for every measured area of the store and every type of customer for every hour and day of week.

DM1’s segmentation capability makes it easy to see whether merchandise is grouped appropriately. How different types of visitor journeys play out. Where promotional opportunities exist. And how and where the flow of traffic contradicts the overall store layout or associate plan. For identified shoppers, it also means you can create extraordinarily rich behavioral profiles that capture in near real-time what a shopper cares about right now.

It comes down to this. Without segmentation, analytics solutions are just baby toys. Segmentation is what makes them real marketing tools.

The Roadmap

DM1 certainly delivers far more segmentation than any other product in this space. But it’s still quite a bit short of what I’d like to deliver. I mean it when I say that segmentation is the heart and soul of marketing analytics. A segmentation capability can never be too robust.

Not only do we plan to add even more basic segmentation options to DM1, we’ve also roadmapped a full segmentation builder (of the sort that the more recent generation of digital analytics tools include). Our current segmentation interface is simple. Implied “ors” within a category and implied “ands” across segmentation types. That’s by far the most common type of segmentation analysts use. But it’s not the only kind that’s valuable. Being able to apply more advanced logic and groupings, customized thresholds, and time based concepts (visited before / after) are all valuable for certain types of analysis.

I’ve also roadmapped basic machine learning to create data-driven segmentations and a UI that provides a more persona-based approach to understanding visitor types and tracking them as cohorts.

The beauty of our underlying data structures is that none of this is architecturally a challenge. Creating a good UI for building segmentations is hard. But if you can count on high performance processing event level detail in your queries (and by high-performance I mean sub-second – check out my demos if you don’t believe me), you can support really robust segmentation without having to worry about the data engine or the basic performance of queries. That’s a luxury I plan to take full advantage of in delivering a product that segments. And segments. And segments again.