Retail execution depends on speed, accuracy, and visibility. Yet many companies still rely on manual audits, delayed reporting, and inconsistent shelf checks that make it difficult to control what is really happening in stores. ShelfMatch^TM helps solve this problem by using image recognition to automate retail audit, improve shelf visibility, and turn store photos into actionable data.

Photo credit: Unsplash | Pipe Gil

For brands, distributors, and retail teams, the challenge is familiar: products can be listed in the system but missing from the shelf, planograms can be broken, price tags can be wrong, and promotional displays can be incomplete. These issues are often discovered too late, after sales opportunities have already been lost. ShelfMatch^TM addresses this gap by giving teams a faster and more reliable way to monitor execution in real time.

Why manual retail audits are not enough

Traditional retail audits are often slow and resource-intensive. Field teams spend time checking shelves, counting facings, recording compliance issues, and filling out reports by hand. Managers then receive this information later, which delays corrective action and reduces the overall impact of the audit process.

Manual methods also create inconsistency. Different employees may evaluate the same shelf differently, and human error can affect the quality of the data. As store networks grow and assortments become more complex, these limitations become even more visible. Brands need a better way to track shelf conditions at scale without increasing the burden on their teams.

How ShelfMatch^TM works

ShelfMatch^TM is a retail audit solution based on image recognition and computer vision. A sales representative or field employee takes a photo or video of the shelf using a mobile device. The system then recognizes products automatically, analyzes shelf conditions, and generates structured audit data.

This approach makes shelf analysis faster and more objective. Instead of relying only on manual entry, ShelfMatch^TM transforms visual information into measurable results. That allows teams to identify execution problems immediately and react while they are still in the store.

Key retail problems ShelfMatch^TM helps solve

ShelfMatch^TM is designed to address the most common challenges in retail execution. These include:

• Out-of-stock detection
• Shelf presence and SKU recognition
• Planogram compliance checks
• Price tag verification
• Void and gap detection
• Share of shelf analysis
• Competitor monitoring
• POSM and promotional display control

These are the areas where shelf execution directly affects visibility and sales. If a product is present in the store but not visible on the shelf, the brand loses opportunity. If a promotion is not executed correctly, the campaign may fail to deliver results. ShelfMatch^TM helps identify these issues earlier and more accurately.

Benefits of automated retail audit

One of the main benefits of ShelfMatch^TM is speed. Teams can collect shelf data quickly and receive results almost immediately, instead of waiting for manual reports to be processed. This shortens the time between detection and correction, which is critical in retail environments where shelf conditions change rapidly.

Another major benefit is consistency. ShelfMatch^TM applies the same recognition logic across all locations, which improves data quality and makes results easier to compare across stores, regions, and retail chains. This creates a stronger foundation for reporting, performance analysis, and operational planning.

ShelfMatch^TM also reduces the administrative burden on field teams. Sales representatives spend less time on manual documentation and more time on actions that improve execution. That makes retail visits more productive and more valuable for the business.

Why image recognition matters in retail execution

Image recognition has become an important tool for modern retail because it converts shelf photos into useful business data. Instead of relying on incomplete notes or delayed inspections, brands can see what is actually happening in the store and act faster.

This is especially important for companies that manage large assortments, multiple store formats, or frequent promotions. The more complex the execution process becomes, the harder it is to control manually. ShelfMatch^TM provides a scalable way to monitor shelf conditions, standardize audits, and support better decision-making.

ShelfMatch^TM for FMCG and retail brands

ShelfMatch^TM is particularly relevant for FMCG companies, distributors, and retail teams that need to maintain strong shelf standards across many locations. It helps track product availability, improve planogram compliance, monitor competitors, and ensure that promotional activity is executed as planned.

For brands, this means better control over execution and fewer missed sales opportunities. For field teams, it means a simpler audit process and clearer priorities. For managers, it means more reliable data and better visibility into store performance.

Conclusion

Retail audit is no longer just about collecting information. It is about making store execution visible, measurable, and actionable. ShelfMatch^TM helps companies do that by using image recognition to automate shelf analysis and improve the quality of retail audits.

If your goal is to reduce manual work, improve compliance, and react faster to shelf issues, ShelfMatch^TM offers a practical solution. It turns store photos into structured data and helps retail teams move from observation to action with much greater speed and confidence.

Contact us to see how ShelfMatch^TM can boost your business.

Retail audits are a critical part of in-store execution, but traditional audit methods often struggle to deliver the speed, consistency, and visibility that modern retail operations require. Image recognition introduces a more scalable approach by turning shelf photos into structured, actionable data. For brands and retail teams, that means better control over compliance, fewer manual bottlenecks, and faster response to execution issues.

Photo credit: Unsplash | Allef Vinicius

The value is not limited to automation. Image recognition supports a more disciplined audit process, where detection, correction, and verification are connected in a single operational flow.

The operational challenge

Retail execution depends on timely information. If a product is missing, a price is incorrect, or a promotion is not implemented properly, the problem needs to be identified quickly. In a manual audit model, however, teams often spend valuable time collecting data, checking shelves, documenting findings, and later interpreting results.

That approach can create delays and inconsistencies. Different auditors may assess the same shelf differently, and managers may receive information too late to act effectively. Over time, these gaps reduce the impact of field activity and limit the organization’s ability to maintain standards at scale.

Image recognition addresses these challenges by standardizing visual audit analysis. It helps organizations move from subjective checks to objective shelf intelligence.

How the process works

The workflow is simple. A field user captures an image of the store shelf, display, or merchandising area. The image recognition engine analyzes the photo and identifies products, shelf positions, facings, price tags, and other execution elements.

The system then compares what it sees against the expected standard, such as a planogram or merchandising rule set. If it finds a mismatch, it flags the issue automatically. This can include missing items, incorrect placement, poor visibility, or non-compliant promotional execution.

When connected to task management tools, the audit outcome can move directly into corrective action. This is where image recognition becomes especially powerful: it does not just report a problem; it helps operational teams address it immediately.

Areas of audit coverage

A strong retail audit solution with image recognition can support many core execution metrics, including:

• Out-of-stock identification
• Product presence on shelf
• Planogram compliance
• Pricing accuracy
• Promotional compliance
• Share of shelf
• Shelf placement quality
• Competitor monitoring

These measures are fundamental to in-store performance. A product may be available in the store, but if it is not displayed correctly, it may still fail to generate sales. Image recognition helps ensure that execution standards are visible, measurable, and consistently monitored.

Why objective analysis matters

One of the strongest advantages of image recognition is the ability to create a consistent analytical standard across locations. Manual audits are influenced by experience, interpretation, and time pressure. Image-based analysis reduces that variability and gives teams more dependable data.

This matters for organizations that need to compare performance across regions, stores, or categories. Reliable data improves decision-making, strengthens reporting, and makes compliance tracking more meaningful. It also gives leadership a clearer view of where operational issues are recurring and where support is needed most.

From audit insight to execution control

The real value of an audit process comes from what happens after the issue is identified. Image recognition is most effective when it is integrated into a closed-loop workflow that supports action and verification.

A typical process follows three steps: identify the issue, assign the fix, and confirm the result. This “see it, fix it, prove it” model helps retail organizations reduce delays and improve accountability in the field. It also ensures that audit findings translate into actual operational improvements rather than remaining as isolated data points.

For retail execution leaders, this creates better control over store standards and a more transparent view of what is happening in the field.

Strategic impact for retail organizations

Retail audit image recognition is more than a technology upgrade. It is a practical way to strengthen store execution, reduce manual effort, and improve the reliability of field data. For large-scale retail operations, the impact can be significant: faster audits, better compliance, more accurate reporting, and quicker corrective action.

It also supports stronger collaboration between field teams, managers, and headquarters. When everyone works from the same visual evidence and the same execution standards, it becomes easier to prioritize actions and maintain consistency across the network.

Closing perspective

Retail audits should not simply document store conditions. They should help organizations improve them. Image recognition makes that possible by giving teams a faster and more dependable way to detect shelf issues, verify compliance, and act on what they find.

For companies focused on retail execution, this approach offers a clear advantage: less manual work, better visibility, and a stronger connection between store observations and business results.

Every day, retailers lose up to 15% of potential sales due to empty shelves, misplaced items, and incorrect price tags. Traditional manual audits often can’t keep up with the pace of change in high-traffic stores. Fortunately, modern technology offers a smart solution – fixed cameras with real-time video analytics.

How does it work?
Cameras mounted above shelves continuously scan product displays. A special AI-powered algorithm instantly analyzes the video feed and detects:

• Out-of-stock situations on shelves;
• Planogram compliance violations (merchandising);
• Mismatched price tags or promotional labels.

The system automatically sends alerts to the store’s management system or a mobile app, enabling staff to react immediately.

Key Business Benefits:

• Instant Reaction: Issues are identified and resolved before customers even notice them.
• Unmatched Accuracy: AI algorithms detect the slightest deviations from set standards.
• Reduced Operational Costs: Optimizes the routes and tasks of merchandisers and auditors by providing targeted alerts.
• Powerful Analytics: Accumulated data helps analyze demand, optimize assortment, and prevent shrink.

This is a scalable solution suitable for both large chains and small businesses. Implementation can start with a pilot zone—for example, with a single product category.
This is more than just “smart surveillance”; it’s a strategic decision-making tool. The technology transforms raw shelf data into actionable insights, helping stores operate more efficiently and increase profitability.

In modern retail, every detail matters – from product placement on shelves to inventory accuracy. This article examines a key aspect of effective merchandising: planogram compliance. Learn how advanced technologies help retailers increase sales and optimize processes.

Planogram Compliance: Why It Matters

What is a Planogram?

A planogram is a “digital twin” of a retail shelf that defines:

• Exact location of each SKU (stock keeping unit)
• Product arrangement by categories, brands and other parameters
• Optimal space utilization based on sales data and customer behavior

Planogram compliance measures how closely the actual shelf display matches the approved plan. This directly impacts conversion rates, average purchase value and customer loyalty.

Market for Planogram Compliance Solutions

• Global market estimated at $1.6 billion by 2030 (13.6% annual growth)
• Fastest growth in Asia-Pacific region (+26.4% annually)
• North America maintains leadership in implementation volume

Business Benefits

• +8.1% profit increase through optimized product placement
• Reduced losses from stockouts
• Unified standards across all store locations
• Real-time data for decision making

Implementation Challenges

• Staff turnover and need for constant training
• Layout differences between stores in the same chain
• Manual compliance checks take up to 30% of working time and don’t guarantee 100% quality

How Technology Solves These Problems

Modern computer vision and AI-based systems enable:

• Automatic shelf display analysis using photos
• Stockout prediction and optimization recommendations
• Interactive reports for suppliers and merchandisers

Shelfmatch is a service for automatic product recognition on shelves that has been working with planograms for several years, providing shelf analytics in just tens of seconds.

Artificial intelligence has revolutionized many industries, but when it comes to precise image recognition, it still lags behind specialized services. Let’s break down why.

1. The Problem of “Training on Generic Data”

Modern AI models (like GPT or Stable Diffusion) are trained on massive but heterogeneous datasets. However, real-world business tasks require recognizing highly specific objects:

• Defects on a production line
• Specific products among thousands of others
• Unique markers (QR codes, labels)

An AI not trained on specific examples will make mistakes.

2. Accuracy vs. “Guesswork”

Dedicated recognition services are fine-tuned for a single task, delivering 95%+ accuracy. AI, on the other hand, often follows a “looks similar, must be correct” approach – leading to errors.

3. Flexibility & Update Speed

If a client needs to recognize a new type of object, a specialized service can be retrained quickly. With AI, it’s more complicated: it requires vast amounts of new data, and even then, high accuracy isn’t guaranteed.

The Bottom Line

AI is a powerful tool for broad tasks, but when you need maximum precision in a niche field, dedicated recognition services remain the best choice.

Need a solution tailored to your needs? We develop recognition service that work exactly how you want.

#imagerecognition

Other Fraud Methods
In this part of the article, we will present theoretical facts: we will describe the main scenarios encountered by us, our clients and partners. Interestingly, merchandisers from different countries use typical fraud methods, so these scenarios can be grouped into several categories.

“It’s Not Me, It’s Him!”
Since field specialists’ work depends on gadgets, software, internet coverage and other force majeure circumstances, there is a strong temptation to shift responsibility for inability to work to external causes. The most common explanations sound like this:

“The app wouldn’t launch/work, so I couldn’t do my job”
“The software drained my phone’s battery and it died”
“Your app froze my phone”
“There was no internet in the store, I couldn’t send anything.”
What can be done:
Track the employee’s equipment status using special software (for example, what battery level they started their shift with). Monitor their actions in the app and the app’s behavior (whether it actually froze).

Location Spoofing
This method is quite old and was used by merchandisers even before retailers implemented image recognition technology. The idea was that specialists would report taking photos in one store while actually being in another. It’s worth noting that such deception was possible until vendors introduced employee geolocation tracking.

In the modern world, this scenario has undergone technological improvement: now field specialists spoof or reset (making real GPS coordinates disappear) their location using apps like “fake GPS” that deceive tracking systems. The motivation for using fake geolocation may be higher scores for visiting certain stores (for example, newly opened ones) or more convenient store locations for the employee.

What can be done:
According to our statistics, about 5% of merchandisers attempt to deceive employers using such programs. The optimal solution currently is continuous monitoring of employee geolocation. Supervisors can see the merchandiser’s movement route between stores with time stamps, thus tracking the movement map and time spent in each store. There is also an alert system where an employee leaving a store early receives a device notification. If they don’t return, a countdown timer starts, and the agent gets another notification that their visit may be marked as potentially fraudulent.

Fake Camera and Object Substitution
Third-party “fake camera” apps are widely available in app stores. These apps have interfaces very similar to real cameras but can’t actually take photos: instead they use fake images (in our case – others’ photos of displays) preloaded to the device storage. Thus, dishonest agents use “fake camera” apps to upload photos of perfect displays from shared databases. Without making any real effort, they get excellent “work” results, high scores and good compensation by the end of the day. Meanwhile, trade marketers remain completely unaware of the actual shelf situation.

What can be done:
Replace the system camera with proprietary software. Develop special algorithms to check image uniqueness. Ban duplicate images in the system.

Staged Photos
This fraud type could be called the most creative. Merchandisers, having material interest in placing certain SKUs (e.g., new products) in premium locations (“golden shelf”, checkout zone), temporarily place SKUs in required spots, photograph them, submit reports, then remove the products from premium placements.

What can be done:
This fraud type is hardest to track, as it’s difficult to programmatically detect photographing one product in multiple store zones, and impossible to prevent merchandisers from moving products.
However, practice shows reduced fraud of this type when changing incentive programs: removing bonuses for premium placements eliminates the motivation.

Low-Tech Fraud
Until now we’ve discussed technology-related fraud. But there’s also “low-tech” fraud, which is quite common. We know cases where an agency hires one employee and gives them two smartphones to do two people’s work, while charging clients for two specialists though only paying one.
Merchandisers themselves sometimes work multiple contracts simultaneously, devoting just a couple hours to each rather than a full workday. This can critically impact merchandising quality in stores.

What can be done:
When possible, prohibit multiple specialists from visiting one store. If this restriction isn’t feasible and clients need multiple reps in one store simultaneously, set other limits: for example, allow hypermarkets but prohibit smaller stores.

#fraud #shelfmatch #recognition #AI #tech

The retail landscape is at a crossroads. As consumer expectations soar and labor challenges persist, a critical question emerges: Can robots solve what humans simply can’t scale? The answer may lie in Nvidia’s groundbreaking robotics initiatives, reigniting a revolution that began years ago—but was derailed by an unexpected disruptor: the pandemic.

The Problem: Precision Meets Pressure

Retail’s Achilles’ heel has long been *inventory accuracy, operational efficiency, and 24/7 consistency. Humans excel at customer interaction, but repetitive tasks like restocking, shelf auditing, and real-time data collection often lead to errors, burnout, and rising costs. Enter *autonomous robots—designed to handle these tasks with surgical precision, freeing staff to focus on what truly matters: the customer.

The Pioneers: A Dream Deferred

Before COVID, companies like Walmart and Amazon led the charge. Walmart famously partnered with Bossa Nova Robotics, deploying shelf-scanning robots to track inventory in real time. These machines roamed aisles, identifying gaps and pricing errors, promising a leap toward the “perfect store.” But the dream hit a wall: Walmart abruptly ended the partnership in 2020, and Bossa Nova, reliant on the retail giant, shuttered operations shortly after.

Lessons learned?
•⁠ ⁠Scalability ≠ sustainability: Early robots struggled with ROI and adaptability.
•⁠ ⁠Over-reliance on single clients can be fatal.
•⁠ ⁠The pandemic accelerated retail’s pivot to survival mode, freezing ambitious tech experiments.

Amazon, meanwhile, doubled down on warehouse automation, proving that some robotics could thrive—but the path was far from smooth.

2025: The Perfect Storm for a Robotic Renaissance

Fast-forward to today. Nvidia’s AI-driven robotics platforms (think Omniverse simulations and Isaac robotics tools) are enabling smarter, faster, and more adaptable machines. These solutions learn from past failures: they’re cheaper, more flexible, and designed to integrate with—not replace—human workflows.

Paired with post-COVID realities—chronic labor gaps, demand for omnichannel agility, and tighter margins—the conditions for a retail robotics reboot have never been better.

Will This Finally Unlock the “Perfect Store”?

Imagine stores where robots:
– Auto-restock shelves before items run out.
– Generate real-time analytics to predict trends.
– Seamlessly blend with human teams to elevate service.

But the ghosts of Bossa Nova linger: Can this generation of robots avoid the pitfalls of their predecessors?

What do YOU think?

1.⁠ ⁠Is the “Walmart-Bossa Nova saga” a cautionary tale or a stepping stone for future tech?
2.⁠ ⁠Will AI-driven robots finally crack the code of the “perfect store,” or are we chasing a sci-fi fantasy?

P.S. If Philip K. Dick were alive today, he might write: “Do algorithms dream of optimized supply chains?”

RetailTech #Robotics #FutureOfWork #AI #Innovation

3) Detecting Photos Re-shot from Screens or Other Photos

In this type of fraud, a merchandiser photographs not an actual product display in a store but an image of the desired display (a photo of a photo). The merchandiser uses the same databases of “correct” product placement images as described in Section 1. Typically, the agent uses two devices—a personal one and a corporate one. They open the target display image on their personal device and then re-photograph it with the corporate device, uploading it into the system “for verification.”

While photos taken from screens were once easily recognizable, advancements in display clarity and smartphone camera quality have made detection much more challenging.

Another source of high-quality images can be printed merchandising guidelines (merch books) that showcase ideal product placements. In our experience, such cases were rare, but this method does exist—even if it seems somewhat absurd at first glance. Here, an employee photographs a printed image and uploads it instead of a real shelf photo. We’ve even encountered cases where merchandisers took screenshots from digital merch books and submitted those.

How We Tackle This Fraud

Our first attempts to train our system to recognize this type of fraud date back several years. At the time, screen-captured photos differed significantly from standard ones. Additionally, we worked with smaller clients, had lower photo processing volumes, and merchandisers were less inventive—making fraud detection relatively straightforward.

However, as technology improved, merchandisers grew more creative, and ShelfMatch™ clients became larger and more demanding. The old “brute-force” training approach no longer delivered the required high accuracy.

The main issue we faced with one of our largest clients was a high false-positive rate. While our neural network excelled at detecting manipulated images, it also frequently flagged legitimate photos as fraudulent. We found that our fake detection accuracy hovered around just 60%.

Since our client wanted not just to detect fraud but also to penalize negligent employees, achieving near-perfect accuracy was critical. While 100% precision is unattainable in real-world data, we needed to get as close as possible.

Our Solution

We researched academic literature and explored various approaches to improve detection.

After studying the problem, we supplemented our dataset with external datasets used in similar fields:

• Rewind Recapture Dataset
• MRGAN for Recaptured Image Detection

This expanded our data pool but did not significantly improve recognition quality—a major hurdle.

Research confirms that neural networks often struggle with domain shift (poor generalization to unseen data). Simply adding external datasets doesn’t help; the training data must closely resemble real-world production images.

Manual labeling of client-submitted photos was an option, but it was too slow to achieve the desired accuracy quickly.

The Breakthrough

We thought: When we lack training images of specific SKUs, we visit stores and take photos ourselves. Why not apply the same logic here?

So, we asked our team to play the role of “inventive merchandisers” and submit fake store display photos.

It worked!

• We added these self-generated images to our training dataset.
• Testing was conducted on real client photo streams.

Results:

• 90% fraud detection accuracy
• Only ~3% false positives

While we continue refining our system, our solution already helps clients detect fraud at scale.

“For every clever merchandiser, there’s an equally clever software developer!” 😊

Need Help Fighting Merchandising Fraud?

If digital fraud detection is a priority for your business, our experts are ready to assist. Contact us—we’ll respond promptly and tailor a solution to your needs.

In the next (final) article, we’ll cover other fraud methods in retail merchandising.

Cases of fraud in digital merchandising are frequent and, alas, inevitable. Field personnel – probably due to the nature of without regular supervision of a manager – are very good at creating new and original tricks to work less and earn more. Therefore, for companies that have merchandisers on staff, the issue of auditing and checking their employees is always relevant.

In this article we will talk about the types of fraud in merchandising and our experience in how to technologically overcome the pitfalls of careless employees. In the 1^st and 2^nd, practical parts, we will dwell in more detail on how we work with certain types of fraud. And in the 3^d part, the theoretical one, to complete the picture we will talk about other popular methods of deception on the part of field personnel.

Fraud in digital merchandising: goals

Fraud is an attempt to cheat the merchandising system in order to influence the result. Since field specialists engaged in taking photos of shelves and creating reports are often paid by the piece, manipulating photographs allows them to create the appearance of a larger number of points of sale visited and shelves captured. Hence, fraud occurs when a field specialist cheats the system to make more money.

Another case is when a specialist’s fee depends not on the number of photographs taken, but on the quality of the work – that is, the KPIs achieved. In this case, the more high-quality photographs a person provides to the customer in terms of ideal shelf layout, the greater reward the person will receive.

In this post we will tell how ShelfMatch^TM works with fake photos, namely:

1. Identifies and searches for duplicate photos
2. Distinguishes and rejects poor quality photographs from the entire stream
3. Identifies photographs taken from smartphone/tablet screen or print media

1) Detecting duplicate photos

Photo duplication is the most common type of fraud. The merchandiser uploads the same photo into the system with an interval of several days or weeks. This photo can be a real photo from a POS (point of sale), or it can be downloaded from a special database: merchandisers all over the world have secret chat groups for exchanging high-quality photographs of shelves, which contain comprehensive gallery of images which could be used in almost any situation. The point of this trick is not to pay a visit to the POS, but to send the supervisor an old, out-of-date photo of shelves with goods – as if the employee had visited the outlet. Consequently, all the merchandisers had amazing results at the end of the day.

How ShelfMatch^TM works with this type of fraud:

On the one hand, finding duplicate images is a fairly well-known, understandable and solved problem for Machine Learning. An image processing service creates a digital fingerprint of the photo, adds it to the database, creates such a virtual fingerprint for each incoming image and compares each new fingerprint with the previous ones, thereby finding duplicate images.

On the other hand, in our context this task is not as straightforward as it might seem at first glance. There are several difficulties that may make work difficult.

First, what is considered a duplicate? It would seem that the answer is simple: exactly the same image of a shelf. But with this understanding of the duplicate, the system will be quite easy to deceive. For example, you can slightly crop the original image, even partially sacrifice some SKUs in the photo. And then, for computer vision, it will no longer be an absolute duplicate – although will be a duplicate, in terms of content. And this is the first difficulty that has to be taken into account when creating a system for finding duplicate images.

Second, for what period should we search through the database of previous visits? This complexity lies in the plane of server capacity. It’s one thing when there is a certain image database of a fixed size. But it’s another matter when this database grows daily by several tens of thousands of pictures or more. As a result, you need to solve the issues of: a) physical space for storing fingerprints and b) speed of comparing each fingerprint with the entire database. You can periodically clear the database of old entries, but sooner or later merchandisers will get a chance to upload older images.

Despite these nuances, ShelfMatch^TM can protect you from persistent and obvious attempts to forge a photo.

2) Detecting bad quality images

It also happens that employees send poor quality photographs: overexposed, blurry, unclear images, photos taken at a random angle, or in which the entire rack is not visible, and other unusable files. All such photos make the automated image recognition impossible. The reason why employees take such photos is because of amount of data. It’s easier to take a lot of low-quality photos and send them to the supervisor than to try to take high-quality photos. And, of course, human factors always play a vital role: everyone can make mistakes and accidentally take a bad photo.

How do we deal with such fraud?

In ShelfMatch^TM a special separate module is responsible for assessing the quality of photographs. Image processing occurs in 2 stages:

1. pre-processing (before SKU recognition) and
2. post-processing (after recognizing the SKU and price tags in the photo).

1. Pre-processing

In the preprocessing stage, the incoming shelf image passes through several neural networks that evaluate following parameters:

• Shooting angle. In particular, the neural network looks for the angle of inclination of the shelves in the photograph. This is a fairly important parameter, since the quality of SKU recognition depends on the shooting angle. The more frontally the photo is taken, the more accurate the recognition results will be.
• Presence of goods in the photo. Another neural network determines whether the photo contains images of goods or has other content. “Other” can mean a lot of options: in practice, we come across photographs from a car, photos of the entrance area of a store, images of the floor/ceiling sent by mistake, and similar irrelevant images.
• Image quality, overall. A separate neural network determines the parameters in combination (“image clarity degree” + “exposure”) and counts a specific value: whether it is an image of poor or good quality.
• Screen photo or real photo. We will talk about this version of fraud in more detail below.

The results of preprocessing allow us to weed out poor quality photos and not to take them into recognition phase, and the merchandiser, in turn, can receive a quick request for a new, high-quality photo (if the work takes place online).

2. Post-processing

It often happens that not the whole photo is bad, but some part of it. ShelfMatch^TM can evaluate such cases using post-processing.

During the post-processing process, the quality of not the entire photo is measured, but separately each small piece of the image containing the SKU or price tag.

During post-processing you can:

• track which merchandisers take perfect or close to perfect photos, and which ones don’t try very hard
• receive additional information about the accuracy/doubtfulness of the SKU and/or price tag recognition results in the specific location of the image.

One way or another, in the field of automated product recognition there is a constant struggle for recognition accuracy. But the recognition accuracy often suffers for reasons beyond the control of neural networks: a large glare crept into the photo, or the hand trembled when taking a photo and part of the image was blurred, etc. In such cases, assessing the quality of the photo, in general, will not give anything. But post-processing and evaluation of individual areas will show that the results from such a photograph are not very accurate and they need to be at least partially omitted. Thus, we have another tool for identifying low-quality photographs, allowing us to increase the accuracy of the analysis.

Besides the question “How to grab the attention of customers and make them want to buy from me rather than from my rivals”, every day, managers, marketers, merchandisers and sales representatives are faced with a large number of other questions – regarding the display of goods on store shelves:

• Are my products placed on shelves uniformly and coherently across multiple stores in accordance with the planogram?
• Are products placed in the order demonstrated in the planogram?
• Are all product items (SKUs) present on the shelf?
• Are product labels facing the consumer?
• Are products surrounded by complimentary items to drive cross selling?
• What is my competitive environment?
• Are all my goods affixes with price tags?
• Has the retailer leveraged promotional shelf labels?
• Are signs, banners, and shelf dividers adequately visible?

And so on.

Timely and accurate answers on the questions above are extremely important to improve the inventory turnover, increase sales volume, skillfully manage items with irregular demand (e.g. seasonal products), and effectively manage contract obligations compliance.

Will you please count the number of SKUs in the photo above in a couple of seconds by eye? How about determining the share of shelf for each product category and comparing the data with a planogram – yep, with your own eyes, but accurately and quickly?

Here’s where Artificial Intelligence will help you out!

In 2,5 seconds AI will recognize and count the number of SKUs, share of shelf, conduct planogram compliance and detect inconsistencies of product layouts. An automated inventory management system can also help businesses keep track of their stock levels automatically with minimal human intervention. How long would it take your sales rep to manually calculate these metrics?

Computer vision in AI is the development of automated systems to interpret images in the same manner as people do. Computer vision technologies are extremely sophisticated and developing quickly. They are capable of coping with merchandising audit (retail execution) tasks not only much more quickly, than a human eye, but also more accurately than the human brain.

Artificial Intelligence and Retail image recognition are coming to store shelves, delivering almost instant access to data captured by field representatives and enabling stakeholders gain smart vital insights into what’s on their shelves and make better evidence-based decisions almost in the twinkling of an eye. 

To be continued.