Augmented Reality

Designing an Augmented Reality Camera to capture and play celebrities in 3D.

View the original post

Brief šŸ“

This design prompt was given to me by the design team at [redacted] e-commerce company.

I came up with my solution called AR Camera (I know, very creative). Iā€™ll be walking you through the case study, explaining how I went around the problem and came up with a solution.

Prompt šŸ’¬

Objective: Designing a volumetric capture and replay solution.
Scenario: The team is looking to bring celebrities, and products into Augmented Reality through a solution called Volumetric Capture.
Challenge: Design a stand-alone application that can playback a volumetrically captured celebrity into Augmented Reality.

Final interactions

Research šŸ§

My first step to understanding the problem was to learn as much as possible about volumetric capturing and augmented reality. This involved me researching journals and watching a lot of YouTube videos to understand how volumetric capture reallyĀ works.

So, what is volumetric capture?

From what I understand, volumetric capture is a new paradigm for designing 3D/Augmented Reality models. Unlike the traditional way, where each and every model is carefully designed by 3D artists, volumetric capture allows computers to do the same thing through cameras placed at variousĀ angles.

Why is this of any significance while designing a standalone application?

Putting in a variety of 3D models just for the fun of it is a bad business idea. Why? Because 3D artists are expensive and doing the same thing for each and every celebrity is just infeasible. Not to mention an e-commerce company is interested in this, meaning theyā€™ll probably integrate the technology to use it to sell different commodities such as, say, furniture.

But now, understanding that the celebrities are volumetrically captured, we can assume that is it NOT infeasible to just go on the app and find your favorite celebrity and capture their ARĀ models.

Competitor Analysis and MarketĀ Research

I quickly found out there are not a whole lot of standalone mobile apps for doing the same thing. The ones that have integrated Augmented Reality cameras in their existing apps include Instagram, Snapchat and whole lot of other social apps. However, all of these apps worked in traditional way and you cannot just go out there and find any AR filter/model you like just by searching.

The prompt specified on building a standalone application. Thus, I had to search for competitors in this space specifically.

So I did find apps in this domain. I downloaded it and started using it. It did not take me a lot of time to figure out what was wrong with some of these apps. Many users had the same problems as well, so I know it was not justĀ me.

Problems with these appsĀ šŸ˜©

I felt the best resource to refer here would be the feedback from the users through Google Play Store and Apple App Store reviews. Filtering useful information from these reviews involved me scouring through hundreds of reviews and coming to a general conclusion about the products available on theĀ market.

Some of these problemsĀ were:

  1. Canā€™t move or rotate! I found out that it is possible to move, scale and rotate. There was just no proper tools to guide the user in moving, rotating and placing the 3D models in theĀ screen.
  2. Models didnā€™t stay put on the floor i.e., there was no way to anchor theĀ model.
  3. Upside down models (yes, Iā€™m surprised too). But in general, the system/application failed to understand the surroundings and just placed the models atĀ random.
  4. No download option. Nonetheless, when I did check out these applications, I did find a download button but it was hidden away in the options and users just werenā€™t able to findĀ it.

With a new perspective on the problems with the apps on the store, I decided to conduct interviews with people. But before that I needed to understand twoĀ things:

  1. Why is this application even needed in the firstĀ place?
  2. Who are the potential users for this application?

Why a standalone augmented reality camera for an e-commerce company?

Without any insights from the company itself, I canā€™t say for sure why an e-commerce company is interested in an AR Camera. But I can make some good assumptions, some of themĀ are:

  • The standalone camera might be a MVP (Minimum Viable Product) to a more impactful, mission critical AR integration within their existing application i.e., customer experience enhancement.
  • The standalone camera is something that doesnā€™t generate revenue directly, but something that is used to enhance the brand name of the company and attract a certain demographic i.e., userĀ growth.
  • Building a different revenue stream by creating a paid model or have a SaaS pricing model i.e., business growth. (not covered / beyond the scope of this caseĀ study)

Who are the potential users?

  1. Avid social media users. Sharing the Augmented Reality video with the companyā€™s branding on social media can help bring new customers onto the companyā€™s platform.
  2. Tech enthusiasts looking to try new products in theĀ market.

Validation šŸ“œ

With a user model, I was able to pick out the right people for interviews. Their pain points validated some of my assumptions. Some of themĀ were:

  1. ā€œModel is randomly placed and I need a lot of time to move and rotate and get it back to where I want it to beā€ (research validated)
  2. ā€œCanā€™t share it with my friends. I donā€™t see the point!ā€ (assumption validated)
  3. ā€œI want to pose/dance with the model but the model keeps movingā€ (research validated)
  4. ā€œAsks for a lot of permissions!ā€ (new painĀ point)

Rephrased problem statement afterĀ research

As an ardent [celebrity] fan (user), I find it hard to pose/dance with [celebrity] through an AR camera which is just frustrating for me to use and I canā€™t even send it to my friends(pain point)Ā later.

Solution šŸ’”

Asking for permissions in context and stating clear reasons for theĀ same

User dropout is usually at a high point when certain permissions are asked in any app. Nonetheless, this number can be reduced by giving clear justifications to why the hardware in question is being used by the application.

Fixing wrong placement of ARĀ models

Loading certain computer vision algorithms to make sure that the AR models are placed not randomly but on the right way in the screen. Of course this will have a buffer time of loading the algorithms and scanning the screen. In the mean time, the app should inform the user of theĀ same.

Transformations of theĀ models

Transformations include rotate, move and scaling the model. While the standard pinch in and pinch out mechanism works well for scaling, putting in rotate and movement in the same screen causes a lot of accidental scaling or rotation.

The anchor tool positions the model on the ground and disables accidental transformations (moves, rotates andĀ scales).

Save and shareĀ videos

A simple interface to share and save the video onto theĀ device.

Complete userĀ flow

Prototype šŸ“±


Behind theĀ Scenes

And thatā€™s a wrap!
I am looking for feedback, so if you have anything to add please mention it in the comments. Also I am open to Product Design opportunitiesĀ šŸ™‚
Thank you forĀ reading!

Did you know you can clap 50 times per article? Click and hold the clap button toĀ try.

Find me on LinkedIn & Portfolio

Designing an Augmented Reality Camera to capture and play celebrities in 3D. was originally published in Muzli – Design Inspiration on Medium, where people are continuing the conversation by highlighting and responding to this story.