Microsoft’s HoloDesk Lets You Play and Interact with Virtual Balls

Who said Microsoft cannot invent? The Microsoft Research team is making use of most interactive and favorite interactivity solution, Kinect, in a research project that will definitely “wow” you. The project is named “HoloDesk“, which allows users to play or manipulate with virtual 3D objects, in an aim to bridge the gap between the real and the virtual world.

The project (system), built by Microsoft Research in Cambridge, is made up of an overhead screen, which has a cabinet like structure, and projects a 2D image through a beam splitter to measure the depth. It uses a combination of projectors, Kinect technology and webcams to read the user’s hands and face. Several custom algorithms are used to calculate the object size, color, distance, and so on, in order to bring everything together in real time.


With HoloDesk you can pick virtual 3D objects, place them on top of the other or even shoot them. For instance, a virtual 3D ball can be picked and placed in a real cup, whose existence, position, size, and other details are recognized by the system.

The Microsoft Research team gave a more scientific explanation –

“A virtual image of a 3D scene is rendered through a half silvered mirror and spatially aligned with the real-world for the viewer. Users easily reach into an interaction volume displaying the virtual image. This allows the user to literally get their hands into the virtual display. A novel real-time algorithm for representing hands and other physical objects, which are sensed by the Kinect inside this volume, allows physically realistic interaction between real and virtual 3D objects.”

The user looks through a transparent display, which is the cabinet like structure, in which holographic objects can be picked up andMicrosoft manipulated with. At present, one can do all these interactions with virtual and real objects within the small cabinet only. However, as technology advances, this limitation could possibly be removed in future.

The HoloDesk project comes from the brains of the folks at the Sensors and Devices team at Microsoft Research Cambridge. However, the technology for sensing motion and displaying images like this isn’t new. An Israeli company called Lumio demonstrated a similar working for virtual keyboards almost a decade ago. But what sets this virtual 3D experience apart from others out there? Here’s what the research team had to say

“For the record, the HoloDesk isn’t the only 3-D interaction experiment out there. But what sets it apart from the rest is the use of beam-splitters and a graphic processing algorithm, which work together to provide a more life-like experience.”

Here’s a video of a user demonstrating the HoloDesk project –

From the video you can see that the interface has some glitches and turn jerky, and the system is far away from being perfectly functional. Nevertheless, it still is something really cool!

Recently, researchers from Carnegie Mellon University and Microsoft Research developed an interesting proof-of-concept called OmniTouch, which is a wearable projecting and sensing device that turns virtually everything into  an interactive touch surfaces. For instance, you can dial a number by projecting a keypad onto your hand or on the wall or on your table. Looks like Microsoft is behind some serious innovations.

Published by

Joel Fernandes

Joel Fernandes (G+) is a tech enthusiast and a social media blogger. During his leisure time, he enjoys taking photographs, and photography is one of his most loved hobbies. You can find some of his photos on Flickr. He does a little of web coding, and maintains a tech blog of his own - Techo Latte. Joel is currently pursuing his Masters in Computer Application from Bangalore, India. You can get in touch with him on Twitter - @joelfernandes, or visit his Facebook Profile for more information.