KinectShop: The Next Generation Of Shopping [Exclusive Video]

A new augmented reality shopping platform for Xbox Kinect will allow users to try on clothes in true 3-D, share photos with friends, and store wish-listed items on smartphones for shopping on-the-go.


Virtual reality shopping just got a lot more real—and could soon become a lot more mainstream. "KinectShop" (working title), an augmented reality shopping platform for the Xbox Kinect, exploits the system's new finger-recognition technology to allow shoppers to grab items from an unlimited shelf of clothes, see how accessories look at multiple angles, and share the photos with friends on Twitter and Facebook for a quick thumbs-up or down.

"The customer can visually see what an object looks like on them without even entering a retail store," Steve Dawson, Technology Director for the Emerging Experiences group at Razorfish tells Fast Company. Unlike existing virtual shopping that shoehorns 2-D photos on top of body snapshots, "with Kinect, you can find the physical outlines of a person and map it to your body."

We'll soon see just how real the augmented reality is, but in the video demo below, the woman using KinectShop looks just like someone trying on clothes in front of a department store mirror, spinning and shifting her body in the familiar fashion poses.

Microsoft's Kinect has been the fastest-selling consumer electronic device in history, has over 10 million owners, and connects nearly 35 million users through Xbox Live—all of whom are capable of online sharing. "This type of experience is really an untapped market because these devices already live at home," says Dawson.

KinectShop is primed to seamlessly integrate with real-life shopping experiences. "With an experience like KinectShop, a shopper can easily scan a QR code or swipe their NFC smartphone to take their experience with them and use wayfinding tools to locate the product in-store," Luke Hamilton, Dawson's Razorfish colleague, writes to Fast Company in an email.

A smartphone's ability to activate apps and alert nearby stores to wish-listed items could automate the time-honored skill of deal-finding, too. Imagine digitally trying on a new purse, getting a round of likes on Facebook, and then being alerted to a big sale at a nearby store while running errands—futuristic shopping, indeed.

New Experiences Required New Tech

The breakthrough grasping technology powering KinectShop was actually built by the Razorfish team. Previous versions of the gesture-recognition technology weren't sophisticated enough to recognize fingers, because hand tracking is one of computer visioning's "biggest challenges," says Dawson. Unlike a face or torso, which maintain the same basic geometric shape, with a hand, "there's so many forms it can take."

The preliminary version that we demoed was a clean experience for simple gestures but still a little buggy: Rotating an open hand to the side activated the grasping command; the algorithm is meant to recognize an open hand by recognizing the negative space between fingers. The final version aims to allow for richer motions, and, in the near future, Kinect is expect to allow single-finger drawing (as was demoed at E3).

So, while we're not quite up to the evolutionary pinnacle of opposable thumb mechanics, we will be able to grab what we like—which is really what shopping is all about.

Follow Greg Ferenstein on Twitter. Also, follow Fast Company on Twitter.

Read More: Most Creative People 2011: Alex Kipman/ Microsoft

Add New Comment


  • Dennis Davis

    Gosh there is just so many advances since the new year its just (IT) astounding. Watching since Pong and the modem its just amazing how far we have gotten, as David Rosen stated "What I am really waiting for is beam-me-up transporter technology.  it has to be close." LOL true, true, David, I want to know when we can fly Tie Fighters LOL.

  • Larry

    This is a game changer - I thought they might do this when I first saw Kenect being used and another video showing some online shopping using some of what Kenect could do.  It was a logical development. Pesonalization for many things that Kenect allows us to do, especially from what I saw on some of the games at E3.  The possibilities for interaction with others and specification to the individual is endless. 

  • David Rosen

    The Kinect has truly leapfrogged the industry as an interface to computing and the internet.  The example showing accessories like purses must just be the beginning. 

    20 years ago, we thought it would improve the shopping experience if you could provide the retailer with your measurements and see what you look like in the garments on your screen.  Lands End rolled out the technology for a while, but I haven't seen it in many years.

    Kinect will revolutionize shopping so that you don't just see a manikin with clothes you are interested in trying, but you will see yourself in the garment...  How cool is that?  And remember, this applique, that the Kinect is just showing the iceberg tip is the second time this is being tried.  It should work the second time around... (Sorry Land's End; you were ahead of your time).

     [The best example is the Cable Industry --  Remember the first attempt was a community putting up its own tower to receive distant signals and then providing wiring to the homes... The second time around, what we have today, is wildly successful].

    Kinect has so many possibilities.  Just watching my 12 year old son moonwalk like Michael Jackson is amazing, but shopping is something that can be monetized.  And think of the other applications for Kinect to help the handicapped interface with computers in any way in which they can talk or gesture.

    What I am really waiting for is beam-me-up transporter technology.  it has to be close.

    David A. Rosen

    Follow me
    on Twitter|

    My LinkedIn Profile|