Apple Vision has been 'in development' for 28 years

Tim Cook once said that “we are high on AR for the long run” and it’s true, for 28 years Apple - and the rest of the tech industry - has been noodling around on augmented reality and virtual reality.

Out of a purely personal interest, I started flipping through rumours about Apple and its “glasses” to see where the leakers got it right and wrong, and the next minute I’m back in 1995, so I thought a curated list of all the leaks, rumours, and related dates might be a nice record to make in the year of our headset, AVP 0.

Enjoy this trip down memory lane:

1981 - Steve Mann designs a backpack-mounted computer to control photographic equipment

While still in high-school Steve Mann wired a 6502 computer (as used in the Apple-II) into a steel-frame backpack to control flash-bulbs, cameras, and other photographic systems.

23 May, 1995 - Apple Technical Report #125: Volumetric Hyper Reality, A Computer Graphics Holy Grail for the 21st Century?

Such a display would convincingly create the illusion of objects with arbitrary optical properties. A metallic object depicted using the display would reflect the visual surroundings of the display. Dielectric materials would show correct refraction and reflection effects. Light shone on the display would illuminate the virtual objects within it. When programmed to depict empty space, the display would, for all practical purposes, disappear, rendering the contained volume invisible. Incremental steps towards such a device are discussed.

1996 - Apple demonstrates a prototype wearable computer system from Apple Computer with a Virtual I/O head-mounted display at the seventh Stereoscopic Displays and Applications conference

September 16, 1997 - Steve Jobs returns to Apple

March, 1998 - Tim Cook joins Apple as senior vice president for worldwide operations

March 21, 2002 - Apple: Stereoscopic Displays?

Apple is said to have other flat-panel technologies cooking in the labs, including stereoscopic displays

Arnold Kim writes:

Stereoscopic displays would presumably simulate 3d/VR environments. A bit unique for the consumer market… but an interesting area of research for Apple.

May 16, 2003 - Simon Greenwold publishes his Spatial Computing masters thesis

June 29, 2007 - Apple releases the iPhone

April 17, 2008 - Apple Researching Laser-Based Head Mounted Display

A user simply plugs their handheld video player such as the iPod manufactured by Apple Computer of Cupertino, Calif., into the compact laser engine attached to their belt, and places the headset on their head. The user then selects a video to be played at the handheld video player (viewing through transparent display elements).

Apple’s patent was originally filed in May 2008 and is based on a provisional patent application filed in May 2007

December 17, 2009 - Apple Working on 3D ‘Hyper-Reality’ Displays

August 24, 2011 - Tim Cook becomes CEO

October 5, 2011 - Steve Jobs passes away

January 12, 2012 - Apple is paving the Way for a new 3D GUI for iOS Devices

The invention covers a 3D display environment for a mobile device that uses orientation data from one or more onboard sensors to automatically determine and display a perspective projection of the 3D display environment based on the orientation data without the user physically interacting with (e.g., touching) the display.

August 2012 - Palmer Luckey launches the Oculus campaign on Kickstarter

22 February, 2013 - Google aims to sell Glass to consumers this year for less than $1,500

26 February, 2013 - John Gruber on Google Glass:

And the idea that people will wear things like this everywhere (as opposed to special specific scenarios, such as workers in an environment where their hands are otherwise occupied, like, say, surgeons) strikes me as creepy as hell.

December 10, 2013 - Apple’s Work on Video Goggles Highlighted in Newly Granted Patent

A goggle system for providing a personal media viewing experience to a user is provided. The goggle system may include an outer cover, a mid-frame, optical components for generating the media display, and a lens on which the generated media displayed is provided to the user. The goggle system, or head-mounted display may have any suitable appearance. For example, the goggle system may resemble ski or motorcycle goggles. To enhance the user’s comfort, the goggle system may include breathable components, including for example breathable foam that rests against the user’s face, and may allow the user to move the display generation components for alignment with the user’s eyes. In some embodiments, the goggle system may include data processing circuitry operative to adjust left and right images generated by the optical components to display 3-D media, or account for a user’s eyesight limitations.

November 24, 2014 - Job Listing Points Towards Apple’s Continued Interest in Virtual Reality

19 March 2015 - Gene Munster Claims Apple Has Augmented Reality R&D Team

May, 2015 - Apple hired Mike Rockwell from Dolby Laboratories

The team, called the Technology Development Group, developed an AR demo in 2016 but faced opposition from then-chief design officer Jony Ive and his team.

May 28, 2015 - Apple Acquires Augmented Reality Company Metaio

January 26, 2016 - Apple CEO Tim Cook: Virtual Reality is ‘Really Cool’, Has ‘Interesting Applications’

It’s really cool

January 29, 2016 - Apple Has Secret Team Working on Virtual Reality Headset

Juli Clover reports:

Hundreds of employees are part of a “secret research unit” exploring AR and VR

February 19, 2016 - Avi Bar-Zeev on his blog: On Holographic Telepresence

I may get around to telling the rest of the story some other time. But I just wanted to say how proud I am of the team and the vision to show the world a glimpse of our collective future.

March 30, 2016 - Microsoft HoloLens released

June 2016 - Avi Bar-Zeev joins Apple

July 26, 2016 - Apple CEO Tim Cook on Augmented Reality: ‘We Continue to Invest a Lot in This’

We are high on AR for the long run

August 23, 2016 - Apple Patent Details Visual-Based AR Navigation Device - reporting on a 2013 patent

The patent notes that visual-based inertial navigation systems can achieve positional awareness down to the centimetre scale without the need for GPS or cellular network signals. However, the technology is unsuitable for implementation in typical mobile devices because of the processing demands involved in variable real-time location tracking.

To overcome the limitation, Apple’s invention uses something called a sliding window inverse filter (SWF) that minimizes computational load by using predictive coding to map the orientation of objects relative to the device.

October 14, 2016 - BuzzFeed News Japan: Tim Cook Talks About Apple’s Augmented Reality Ambitions

November 14, 2016 - Mark Gurman: Apple Said to Explore Smart Glasses in Deeper Wearables Push

January 9, 2017 - Robert Scoble on Facebook: “Apple and Zeiss working together on augmented reality optics”

January 17, 2017 - John Gruber spitballing Apple AR’s usefulness

January 31, 2017 - Apple patents detail augmented reality device with advanced object recognition, POI labelling

February 28, 2017 - Apple Exploring AR in Israel as Robert Scoble Insists ‘Mixed Reality’ Glasses Coming This Year

June 26, 2017 - The Verge reports “Apple’s AR is closer to reality than Google’s”

Apple has often been accused of acting like it invented things that others have been doing for years. That complaint is not without merit, however, Apple can lay claim to transforming existing things into mainstream successes, which takes no small amount of invention in its own right.

June 29, 2017 - John Gruber on Genre Munster’s Apple Glasses predictions

I’m hard-pressed to think of anything we do today on our phones that would be better using AR glasses. Anything.

August 4, 2017 - Apple Experimenting With Several Augmented Reality Glasses Prototypes

November 2, 2017 - Tim Cook on Augmented Reality: ‘What It Will Be, What It Can Be, I Think It’s Profound’

December 4, 2017 - Apple Supplier Quanta Computer Teams Up With Lumus to Make Lenses for Augmented Reality Smart Glasses

January 12, 2018 - Apple Reportedly Met With Potential Suppliers of Augmented Reality Glasses at CES 2018

During CES, representatives from major players like Apple, Facebook, and Google met with suppliers that make the nuts and bolts required to power AR glasses, according to people familiar with the meetings.

March 1, 2019 - Tim Cook to Investors: Apple is Working on Future Products That Will ‘Blow You Away’

February 4, 2019 - Variety: The Inventor of the HoloLens Just Left Apple

Bar-Zeev has been working in the AR/VR space for close to three decades. Back in the ’90s, he was part of a team at Disney that worked on some early VR experiences for the company’s theme parks, including “Aladdin’s Magic Carpet” VR ride.

He then went on to co-found Keyhole, the company that later got acquired by Google to become the foundation of Google Maps. After a brief stint at Linden Lab, Bar-Zeev worked for four years at Microsoft.

“He helped found and invent Hololens at Microsoft, assembling the very first AR prototypes, demos and UX concepts, sufficient to convince his leadership,” according to his LinkedIn bio.

Avi says:

“I left my full-time position at Apple in January. I had the best exit one can imagine. I have only nice things to say about Apple and won’t comment on any specific product plans.”

March 8, 2019 - Kuo: Apple’s AR Glasses to Launch in 2020 as iPhone Accessory

June 27, 2019 - Jony Ive leaves Apple.

August 29, 2018 - Apple Purchased Akonia Holographics, a Company That Makes Lenses for AR Glasses

November 11, 2019 - Apple Said to Release AR Headset With 3D Scanning in 2022, Followed by Sleeker Glasses in 2023 & New AR Sensor Coming to 2020 iPad Pro and iPhone Models, AR/VR Headset as Soon as 2021

Plus, The Information reports on the infamous “1000 person meeting”: Apple Eyes 2022 Release for AR Headset, 2023 for Glasses

Apple executives discussed the timelines, which haven’t been previously reported, in an internal presentation to employees at the company’s Cupertino, California, campus in October, according to people familiar with the matter. Apple Vice President Mike Rockwell, who heads the team responsible for Apple’s AR and virtual reality initiatives, led the meeting, which included new details about the design and features of the AR headset, these people said. The product timetables run counter to recent analyst and media reports that said an Apple AR device could arrive as early as next year.

March 24, 2020 - Apple’s AR Glasses Could Launch by 2022 as Suppliers Reportedly Ramp Up Development

May 19, 2020 - ‘Apple Glass’ Rumored to Start at $499, Support Prescription Lenses, and More

Apple originally planned to unveil the glasses as a “One More Thing” surprise at its iPhone event in the fall, but restrictions on in-person gatherings could push back the announcement to a March 2021 event

May 21, 2020 - Jon Prosser Claims Apple is Working on ‘Steve Jobs Heritage Edition’ AR Glasses, Gurman Calls Rumor ‘Complete Fiction’

They’re also working on a prototype, a Steve Jobs Heritage Edition, similar to how we had an Apple Watch Edition, like that ridiculous $10,000 gold one when it first came out. Some like tribute to Steve Jobs, obviously just like a pure marketing ploy at this point.

Extra: Twitter thread between Prosser and Gurman on the subject.

May 21, 2020 - Apple’s Augmented Reality Glasses Again Rumored for 2021 Launch

October 22, 2020 - Apple Glasses Will Reportedly Use Sony’s ‘Cutting-Edge’ OLED Micro-Displays to Deliver ‘Real AR Experience’

January 6, 2021 - Apple Glasses Reportedly Progressing Towards Engineering Verification Stage With Focus on Battery Life and Weight

January 21, 2021 - Bloomberg: Apple’s First AR/VR Headset ‘Pricey, Niche Precursor’ to More Ambitious AR Glasses and Could Launch Next Year

March 7, 2021 - Kuo: Apple to Launch Mixed Reality Headset in Mid 2022 and Augmented Reality Glasses by 2025

April 26, 2021 - Apple Glasses Prototype Reportedly Falls Behind 2021 Testing Schedule

October 28, 2021 - Facebook rebrands to Meta, as in “the metaverse”

Mark Zuckerberg: