Patent shows Apple is working on improving iPhone’s AR capabilities

Man using augmented reality to look at machinery

A newly granted patent shows Apple is looking to improve how iPhone cameras capture three-dimensional detail, which could result in more immersive displays of augmented reality (AR).

The company’s new patent, first reported by Apple Insider, relates to how an iPhone captures 3D images and collect sdata for an AR walkthrough experience.

Currently, iPhone cameras capture the intensity of light in the scene they’re photographing. According to the new patent, future iPhone cameras would also capture the direction the light travels while recording the relative location of objects in the scene.

“The images and information may be processed to determine metadata including the relative positions of the images and depth information for the images,” the patent says.

“The light field panorama may be processed by a rendering engine to render different 3D views of the scene,” the patent says, “to allow a viewer to explore the scene from different positions and angles with six degrees of freedom.”

The “six degrees” reference means that, within an AR display, the viewer could move forward or backward, look up or down, and even turn around and see different views of the same scene.

“Using a rendering and viewing system such as a mobile device or head-mounted display,” the patent says, “the viewer may see behind or over objects in the scene, zoom in or out on the scene, or view different parts of the scene.”


IT Pro 20/20: What the EU's new AI rules mean for business

The 17th issue of IT Pro 20/20 considers the effect of new regulations on the IT industry


Apple has shown plenty of interest in AR. Not only has CEO Tim Cook described it as having "profound" potential, but Apple also recently purchased AR startups Akonia Holographics and Metaio and applied for many other AR-focused patents.

There are also rumors Apple is developing an AR headset to market as an iPhone accessory. The headset would likely act as an additional screen, projecting documents, videos, games, and more to present the virtual equivalent of a super-sized display.