Camera is the single most common and probably the most powerful people-measurement technology currently deployed. And the most common type of camera for people measurement is a stereoscopic, ceiling mounted video camera that is also true IoT device. It has an on-board processing unit to do the heavy lifting of identifying people, positioning them in the frame, and then tracking them over time. This kind of camera doesn’t send a video signal anywhere. Only the data generated by the onboard processor – random person id, timestamp, position and whatever other events – like line crossings – that are being generated are sent to the cloud.

The best cameras in this class are really good at people identification. In reasonable conditions (normal light and unobstructed), they will typically deliver better than 99% accuracy for door-counting  – making them the champions when it comes to reliable person counting data.

As is usually the case in this world, you pay for that accuracy. Good cameras are expensive and camera is almost always the most expensive way to measure people. But for demanding applications like Occupancy, it’s almost always worth it.

Aside from expense, there are a few issues when it comes to camera tracking. As you’d probably expect, video cameras don’t work well in poor light situations or in highly obstructed environments. What the camera can’t see, the camera can’t measure. In addition, it turns out that measurement cameras have highly-optimized ML for door-counting. But door-counting is just one application of people measurement. Here at Digital Mortar we often use these cameras for queue, for display measurement and for full journey tracking. That highly-optimized ML is sometimes TOO highly optimized and let’s us down in non door-counting applications. Finally, cameras are challenged when it comes to very large areas – and not just because they are expensive. While it’s possible to combine many cameras together into a single field of view, there are limits and issues everywhere when you do this.

Variations on Camera Technology

The basic IoT measurement camera covers a lot of people-counting situations. We use it for door counting, occupancy management, queue measurement, some merchandising analytics, and both labor and shopper journey tracking. But it doesn’t cover every situation. In some cases, that’s just because camera isn’t a good solution (measuring repeat visitors is a good example). In other cases, though, there are camera variants worth understanding.

Poor lighting is one of the most obvious limitations on camera performance. Low lighting is the most obvious example, but very bright environments, wildly shifting light environments, and very cluttered environments can all defeat normal people-counting video cameras.

There are camera variants that work in almost any conditions though. Time of Flight sensors send out pulsed beams of light (usually infrared) and then measure their time to return. This technique is pretty much like radar. It creates a mapping of a scene and then identifies moving objects in the mapping.  Time of Flight systems are usually more expensive than people-counting systems and are not as turnkey – so they introduce cost into every level of the system. Nor are they as good for things like merchandising analytics. But if you need people tracking in challenging conditions, TOF might be the way to go.

Another limitation of measurement cameras is that they are essentially a black box. Since they aren’t pushing video to the cloud, you can’t customize the analytics at a deep level. We use the lowest-level data feeds from measurement cameras, but it’s still just people/position data. So you can’t use measurement cameras for ANY customized video analytics. No face recognition. No eye-tracking. No customized interactions. No object recognition. Unless the vendor has pre-built something into the camera, you CAN’T do it with a measurement camera.

What can you get pre-built? Gender identification. Mask detection. Safe distancing (which is mostly useless). Height. Cart detection (which is kind of buggy). That’s about it.

If you need to do more, you’ll likely have to piggyback a simpler video camera with a more powerful open processor. Scrapping the dedicated measurement camera for a cheap digital video camera connected to a single board computer running video ML is less turnkey and usually less accurate but opens up a range of video processing applications. We typically use Nvidia’s Jetson line of computers if we’re going in this direction

The last variant/limitation to be aware of is that cameras are sensitive to both angle and height. In general the higher your ceilings the bigger your field of view. But most camera variants will only support up to some maximum ceiling height and it’s often more expensive to get models equipped for higher ceilings. Cameras are also limited in how much they can be angled – which is frequently a problem when trying to cover outdoor spaces.

Outdoor and angles variants of measurement cameras do exist and, you guessed it, generally cost a little more.

Matrixing and Stitching

The overwhelming majority of measurement cameras are used for one purpose – door counting. Not surprisingly, that’s what they are best at. But at Digital Mortar door counting is not our only focus. We do it as part of full store measurement, but most of the work we do is focused on journey tracking, merchandising analytics and queue management.

What we’ve found is that the onboard smarts of measurement cameras are often poorly optimized to meet those needs. Sometimes their data feeds aren’t even setup in ways that give us the information we need. And even when the feed can do the job, the actual data provided by the IoT camera is often flawed.

There are three issues that you need to understand if you’re thinking about using IoT cameras directly (though if we’re doing the work you don’t have to worry about these).

We call the first problem ghosting. Ghosts are short data trails that don’t actually represent any person. They can be created by all sorts of things – from swinging doors to balloons to box carries to tricks in lighting. We see them in EVERY system we work with and they are NOT infrequent. If you’re using the pre-canned events like line-crossings in a door-way, ghosts aren’t a big issue. Vendors usually have logic in their systems that keeps them from tripping events and doorways are usually very clean environments. But camera vendors don’t generally deploy that logic in the raw feed and in complex environments the data can get messy. So if you’re using a lower-level object feed it can be pretty scary. You’ll have to filter ghosts to get remotely accurate data.

The second problem with IoT cameras is related but worse. When we use cameras for journey and zone tracking we want to understand how a single person object moves through the camera frame. Unfortunately, IoT cameras OFTEN lose track of people for short amounts of time. So what do they do? Mostly they just create a new person object. Again, this doesn’t tend to do much to their pre-packaged line events, but it actually does whack out things like their zone dwell and dwell time metrics.

We’ve written custom logic to stitch these visitor fragments back into a single continuous user session with a single ID. Without that, every application doing more than line crosses is compromised. This logic, by the way, isn’t straightforward. In even moderately crowded environments it’s tricky and almost certainly imperfect. We’ve spent countless hours optimizing this kind of stitching problem.

Highly related is problem number three – matrix stitching. If you want to track within a single camera zone you have to stitch fragments together. But what if you want to track across larger areas? We do this all the time. Most IoT measurement cameras come with the ability to matrix multiple cameras into a single field of view. Usually, you can only do about 10 cameras in a matrix before you have to buy something extra (or are just plain out of luck). Matrixing cameras actually works pretty well. There isn’t any kind of people recognition going on – the camera fields of view have to overlap and the matrix is literally using a geometric overlay to map people from one camera field of view to the next. But properly setup, it works reliably.

Still, stitching problems are even more severe in matrixed views. So in matrixed situations, we have do stitching both inside the single field of view and sometimes between arrays of matrixed cameras. It’s a bear.

Bottom line – measurement cameras aren’t quite the generalized people-tracking solution they might appear to be. They are very good at people identification. In most cases, they are the right technology solution. But picking a variant that fits your situation is important, and so is doing the work necessary to clean-up the raw data if you’re trying to do more than door-counting,.