Augmented Reality: new standards and new tools

Thanks to their sensors, mobile phones have become powerful platforms for augmented reality. Marius Preda and his team of researchers from Télécom SudParis, GRIN (Graphics and Interactive Media), have developed technologies recently chosen by ISO (International Organization for Standardization) for the production and transmission of augmented reality content.

As their capabilities have developed, smartphones and tablets have become powerful platforms for 3D visualization. The integration of sensors (video, photo, position, etc.) has made them platforms for augmented reality. “The idea is to build a bridge between the digital world and the physical world, by enabling the identification of elements in the real world that will trigger a digital action on a smartphone or tablet, and provide a combined view on the screen,” explains Marius Preda, a researcher at Télécom SudParis, in charge of the “graphic content” group for MPEG, a media standardization organization within ISO. His team creates standardized content production and transmission tools for augmented reality, for use with mobile devices.

Formal standardized language for augmented reality

Most companies are currently developing integrated augmented reality applications, but another development method is also possible: “We can formalize augmented reality experiences using a standard language that defines the user’s experience and can be interpreted by a browser, just like the web works today.”

Marius Preda and his team therefore developed ARAF technology: an open-source navigator with its own formal standardized language. “Mobile telephone manufacturers will have their own ARAF installed and optimized for their platforms, and content producers will be able to quickly create augmented reality experiences using authoring tools, without going into all the technical details, just like today we can create a website without knowing HTML.”

An image recognition cloud platform

At the same time, researchers have developed an image recognition cloud platform within a database. A unique signature, which is specific to each image, makes it possible to compare images and find similar objects in them, such as the Eiffel Tower, for example. To avoid false positives and false negatives, the development of the algorithm must involve a compromise between these two problems.

“One way of doing this is to identify unique characteristic points: we reproduce the human visual system, which focuses its attention on control points with many details,” explains Marius Preda. We obtain a constellation of unique points for an object and we record characteristics, such as the distance between the points, which will be used to compare an image with others in the database. We also work on atypical cases by using contour detection, especially to differentiate cars using images of their tail lights. If the signatures are similar, there’s a big chance the images are similar as well.”

This technology was used in particular by the GOOT application to recommend a similar wine, based on a picture of a wine label.

Authoring tools facilitate the creation of augmented reality content

Based on this technology, researchers are also creating authoring tools, like those developed as part of the BRIDGET European project. For users, these tools make it possible to offer quality, interactive and customized content on a second screen while watching television. A signature audio system enables the tool to recognize the program being broadcast, downloads additional content from a server, and displays the content in a synchronized manner.

For content producers, a video on the main screen can be enhanced, either manually using the web, or automatically using a visual search system: “With our huge database of indexed and annotated images, we try to see whether the content is similar to that in the database for each video frame,” explains the researcher. For example, if the Eiffel Tower is recognized in the video, the tool will offer information on the monument that is indexed in the database or retrieved from web. A second phase of the project plans to integrate a connection with social media.

There are many uses for these tools (personalized advertising, interactive cultural content, etc.) and many that have yet to be imagined, which is why companies are encouraged to try the technology and contribute to the database on the cloud platform via APIs.

Learn more about MPEG group