Augmented reality is digital content positioned onto a live camera feed which makes it a part of the physical world. AR overlaps computer generated video onto camera feed so cleverly that the CGI objects appear to have a real location in this world. The experience may also include some or many perceptual elements like hearing, touch, feel (like warm or cold) and smell. Let us dive deeper and see how augmented reality works.
Most people know AR thanks to Pokemon Go. It was a big success with 45 million users enjoying it in 2016.
In 1968 Ivan Sutherland also known as the father of computer graphics developed an AR head mounted display for use in aviation, military and industrial purposes.
Only in 2008 in a marketing show by BMW the application was demonstrated commercially where one could move around a BMW mini car model printed on paper and filmed by a computer camera.
The humble phones and tablets are the way augmented reality gets into most people’s lives. Vito Technology’s Star Walk app allows a user to point the camera in their tablet or phone at the sky and see the names of stars and planets superimposed on the image.
Similarly the app called Layar uses a smartphone’s GPS and its camera to collect information about the user’s surroundings. It then displays information about nearby restaurants, stores and points of interest.
Mobile games like GO, Harry Potter: Hogwarts Mystery have been keeping the public informed about the technology and what it can do.
There is also Google Glass which is a headset that you wear like a pair of eyeglasses in return it keeps you constantly plugged in to your e-mail, calls and other notifications.
There are also neurosurgeons that use an AR projection of a 3-D brain to aid them in surgeries. Ground staffs at Singapore’s airport wear AR glasses to see details about cargo containers there by speeding up loading times.
Furniture giant IKEA has an AR app called IKEA Place that allows you to see how a piece of furniture will look and fit in your room.
How it works?
- The camera supplies real world video to the display and graphics algorithm.
- Registration gives the computer reference on where to place the virtual objects. High-contrast icons are a quick and dirty registration tool; but GPS, accelerometers, orientation sensors, and barometric sensors can be more robust. Landmarks like road lines, buildings, and the horizon can also provide alignment details.
- Computer vision understands the footage and determines how to integrate virtual objects. Some AR CV is just relative placement over a registration mark, while more advanced algorithms can do shadows, occlusion (things in front of the virtual items), and kinematics (physics and interaction with real-world objects).
- Then it streams the combined footage all back to the user. A computer monitor makes an AR “terminal”, a cell phone appears like a magic mirror into the AR world, and a wearable display can provide total immersion.
How can you develop one?
If you want to develop augmented reality, the go to library is the OpenCV. Other derivatives like FLAR and ARToolKit may be more useable. Similarly Unity is a framework purpose-built for AR development that allows you to develop your app and deploy it across multiple mobile and wearable AR devices.