VRHow / VR History / The development of mixed reality

The development of mixed reality

Short answer: Mixed reality did not begin as one standardized product category. It grew from tracked, see-through research displays into industrial augmented-reality tools, phone-based AR, optical-see-through smart glasses and, more recently, consumer headsets that reconstruct the room with cameras. “Mixed reality,” “augmented reality,” “spatial computing” and “passthrough” are era- and vendor-dependent labels, so the display path matters more than the marketing name. Last checked October 11, 2026.

What the terms mean

Optical see-through leaves the real scene visible through a transparent combiner or waveguide and adds computer light to it. Video see-through (often called passthrough in current products) blocks the direct view; cameras capture the scene, software combines that video with rendered graphics, and displays show the result. A technical comparison by Rolland, Holloway and Fuchs describes the tradeoff: optical systems preserve an essentially unhindered, immediate real-world view, while video systems provide more control over composition and occlusion but add camera, processing and latency constraints. [technical comparison]

The words changed as products changed. “Augmented reality” usually described adding registered information to the real world; Microsoft used “mixed reality” for HoloLens and Windows Holographic; Apple used “spatial computing” for Vision Pro; Meta calls Quest 3 a mixed-reality headset. Those labels describe overlapping experiences, not a stable taxonomy. This article therefore identifies the optical or video path where the evidence allows it, rather than treating a vendor label as a technical definition.

1960s–1980s: tracked displays before the category had a name

Ivan Sutherland and Bob Sproull’s 1960s head-mounted display is a foundational milestone. The Computer History Museum dates its pictured artifact to circa 1967 and records the large overhead support that led to the nickname “Sword of Damocles”; its wire-frame room changed with the user’s head movement. The later 1968 paper is commonly cited as the system’s publication date, so “1967–1968” is safer than claiming one uncontested first date. [Computer History Museum record] [1968 paper record]

This early system belongs in mixed-reality history because it was see-through rather than a sealed modern VR visor: the research problem was to register computer-generated imagery with a user’s view while tracking the head. Later researchers continued both optical and video approaches. In the 1994 comparison paper, UNC researchers describe optical combiners and camera-based video see-through as two ways to superimpose synthetic imagery on the real scene. [UNC technical history]

1990s: industrial AR makes registration useful

Boeing researchers Thomas Caudell and David Mizell used the phrase “augmented reality” in their 1992 paper on a heads-up, see-through head-mounted display for manual manufacturing. The proposed system placed wiring information over the worker’s task, tying the display to an industrial workflow rather than a general-purpose virtual world. Because the paper is the primary record for the terminology and prototype, it is stronger evidence than later “first AR” lists. [IEEE record for Caudell and Mizell, 1992]

Other 1990s research explored maintenance, collaboration and tracking. A peer-reviewed historical review records Rosenberg’s 1992 “Virtual Fixtures,” the 1993 KARMA maintenance project, collaborative Studierstube work in 1996 and an outdoor tracked system at Columbia in 1997. These are research milestones, not evidence that a consumer product category already existed; “first” wording in retrospective histories is often dependent on the chosen definition. [peer-reviewed historical review]

2000s–2010s: from toolkits and phones to consumer optical AR

Marker tracking and open software helped move AR beyond bespoke laboratory rigs. The same historical review dates matrix-marker work to 1998 and the first ARToolKit publication to 1999, while noting later industrial maintenance applications. These milestones mattered because they made registration and prototyping more repeatable, even though they did not settle whether “AR” meant optical display, camera preview or projection. [historical review]

Google Glass brought a lightweight, limited optical display to public attention. Google’s own project history says its team was wearing prototypes in public by April 2012 and opened the Glass Explorer Program in April 2013; it later shifted toward Enterprise Edition, launched in May 2019, for hands-free work. Glass was not a full-room 3D mixed-reality headset: it illustrates how a small optical overlay and camera can be marketed as wearable computing without matching the capabilities of later spatial systems. [Google/X Glass history]

Microsoft announced HoloLens in January 2015 and shipped developer and commercial devices in the United States and Canada on March 31, 2016. Microsoft called it a self-contained “holographic computer” and positioned Windows Holographic as a mixed-reality platform. HoloLens represented the optical-see-through branch: a transparent display placed holographic content into the user’s direct view, rather than showing a camera reconstruction of the room. [Microsoft announcement]

2016–2020s: AR reaches phones, then passthrough reaches headsets

Smartphone AR made the idea familiar without requiring glasses. Pokémon GO’s 2016 release is a useful consumer milestone, but it should not be confused with an optical-see-through headset: the phone camera and screen form a video-see-through window. The broader historical point is that consumer AR adoption could scale through existing phones even while wearable displays remained expensive and technically constrained. [BBC account of Pokémon GO’s AR model]

The newer consumer path is video passthrough. Meta’s official September 27, 2023 Quest 3 post announced an October 10 retail date; an October 9 update recorded the headset as on shelves. Meta described Quest 3 as a mass-market mixed-reality device with full-colour passthrough, letting software place virtual content over a camera view of the room. That is technically different from HoloLens: the wearer sees a reconstructed camera feed, not the room directly through transparent optics. [Meta’s announcement and launch update]

Apple Vision Pro extended this camera-based approach into Apple’s “spatial computing” vocabulary. Apple says its camera system transmits more than one billion pixels per second to the displays, and its R1 processor streams camera and sensor input to the displays; Apple announced U.S. availability for February 2, 2024. The product can blend digital content with the physical world, but its view of that world is video passthrough, not optical see-through. [Apple technical overview] [Apple launch announcement]

What changed across the timeline?

The durable progression is not simply “VR became MR.” It is a sequence of engineering tradeoffs: mechanical tracking became electronic inside-out tracking; bespoke wireframes became registered industrial instructions; small overlays and phone screens made AR familiar; and cameras, depth sensing, display panels and computer vision made room-scale video passthrough practical for consumers. [AR history] [Quest 3] Optical see-through still has the advantage of direct real-world vision, while video passthrough can control occlusion and composition more completely. [see-through comparison] Neither is universally superior, and a vendor’s use of “mixed reality” does not identify which one it uses.

Sources