With the 3-hour long Endgame approaching the theaters, I went to a recently inaugurated supermarket to buy an Avengers’ T-shirt, you know, to show that I am like a more-than-normal fan (which obviously dozens of people already are anyway). All went good except that I wasn’t able to find any! Let’s see if AR can help me out.

Not that my desired shirt wasn’t in stock, but because I was unable to find the section of that supermarket which had clothes on display. Two minutes later, and conversing with 3 of its employees, I was finally able to find the one, the one that would make me prepared for the epic finale of Endgame, although I still wasn’t prepared for what came in the end, but let’s leave that for another day and … move on.

Got the shirt, finally!

Coming to the problem

It was then that it came to my mind, that this problem might be even more existential for people who are shorter. I am tall enough to sometimes overlook the shelves (being 6 ft), but not everyone is. And the same problem would be faced by the handicapped people, who might need assistance from the attenders of that supermarket to get to know about the location of a section.

So the next obvious question is, how to solve this one?

AR to the rescue!

This is where the Augmented Reality experience can help us with. A quick search on Google gives its definition as:

“a technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view."

Google

The main concept is that we need to somehow augment the real world experience of a supermarket with information, and with AR in an app, we can bridge this gap (See what I did here? ) .

Implementation

With the end-goal (endgame?) in mind, we thought about the various ways this could be achieved. After all the brainstorming, we decided to break it down into 3 steps to get started with:

  1. Get the geofence coordinates of the supermarket (to identify if the customer is within the store or not)
  2. Get info & geofence coordinates of various sections in that store (so that later it can be related with user’s current location)
  3. In the app, get the user’s current GPS location and then process all the sections w.r.t. the coordinates of each section’s geofence.

Step 1 and Step 2 are basically the data collecting steps, which is stored in either the app or can be stored in back-end for multiple outlets and stores. A visual representation of the collected data will be as follows:

A (literal) blueprint for the visual data that need to be stored

Step 3 is where the actual magic happens. Whenever customers enter a retail store, what they need to do is that just open the app on their smartphone, and with camera access, lift and point the phone in any direction. The mobile app, which will have data of stores and their sections, and also the user’s current location, can make an approximate guess about where the user is currently present, and what all sections are near or away from them and in which direction.

A well implemented version of this might look like this:

Concept of how the app’s viewfinder will be showing

Allowing ordinary mortals like us, to see-through store shelves 😉

At any point, the app will decide what to show on screen and where to show it on screen based upon the data which was stored before, and the relativity of it with the user’s current location.

And there we have it : ShoppAR – Augmented Shopping Experience!

Future scope

With the basic functionality now clear, some side-features that can complement this might include:

  • show an ad in AR mode, when a poster or code is scanned on the walls/shelves of the store
  • show operating instructions of various products, such as to show how the drone on the shelf needs to be assembled before operating
  • show simulation of how the product would look like with its dimensions in real life
  • show discounts on a per-shelf basis in an augmented way over the real feed of the shelf from the camera
  • in-app payments can also be added to checkout from the stores without going through the line

This is barely a scratch. With the enabling of Augmented Reality experiences on our extremely easy to carry mobile devices, a vast sea of possibilities exist, each one solving a problem that people won’t even realize can be solved with just this! But if you do find an extremely awesome way in which AR and/or other technologies can solve a problem, have our team hash it out for you and make a real product out of it.

As for the Endgame, well it was 10 on 10! But it being an “endgame" for some of my favorite characters was actually upsetting. But then I remember what Vision said, and I quote – “A thing isn’t beautiful because it lasts".

can you buzz me in?

Prafull Mishra

Android Developer at Terasol Technologies
Hi! I am an Android Developer at Terasol Technologies. I convert coffee to mostly Java and Kotlin code. Also responsible for designing of apps and backend services.
Prafull Mishra