Tracking Influential Augmented and Virtual Reality Patents In Gaming

Despite the coronavirus pandemic and perhaps a little because of it, consumer demand for virtual experiences and online gaming continues to accelerate R&D efforts in these areas. We examine particularly augmented reality (AR) and virtual reality (VR) patent landscapes.

Consumer consumption of media has increased dramatically in recent years with the growing popularity of mobile gaming and availability of new devices and services. According to one study, mobile gaming contributes to 26% of all media consumption hours (Source: Ericsson). One interesting aspect of this study is that while 95% of gamers have an interest in AR/VR gaming, which is not entirely unexpected, 36% of the non-gamers have also shown an interest in AR/VR gaming. The prolonged quarantine period is expected to further drive the demand for immersive experiences.

Virtual Reality (VR) provides a more immersive experience as it isolates the gamer from the real world while providing a varied level of engagement with a virtual world. VR gaming can be experienced today through a variety of basic and advanced VR-capable Head Mounted Display (HMD) devices, such as Facebook’s Oculus, HTC Vive, PlayStation VR, FOVE and others. The latest devices feature high resolution (4K & 8K) and high refresh rate displays, spatial audio, improved haptics, low latency and high bandwidth communication networks (e.g. 5G, WiFi-6E, etc.). Despite the availability of capable devices in the market, VR devices have not taken a considerable market share of the gaming market. Lack of interesting content, expensive hardware and some other limitations, such as restrictive mobility while gaming and user isolation have been cited for holding back VR from becoming a mainstream choice (Source: link).

Augmented Reality (AR) allows for real-world interaction while providing an integrated immersive experience similar to that of VR. Mobile AR gaming alone is expected to generate more revenue than VR by 2021. Mobile AR gaming provides social engagement and immersive experiences, without any need for specialized hardware. It can be implemented right from a user’s smartphone. Apple’s ARKit and Google’s ARCore has reached a combined 117 million users per month together supporting more than 3,000 applications. This is best illustrated by the popularity of Pokémon Go, one of the highest grossing smartphone games in the world.

The number of new patents related to AR/VR in gaming filed each year increased globally by 33%, on average, starting from 2010 (Figure 1). This exponential rise in the number of filings is one indicator of increased R&D spending in this domain for each year.

*forecasted Data Figure 1: Application trend for worldwide patent/applications related to AR/VR in gaming

Of the total number of patents related to AR/VR in gaming, almost 50% have been assigned to or owned by the top 25 assignees. This may be an indication that the top players have a strong foothold as a result of their R&D efforts or through consolidation efforts, i.e., M&A. It is reasonable to conclude from the above the top assignees are continuing to bet big on the AR/VR domain application to gaming.

Figure 2: Top assignees based on their worldwide patents/applications related to AR/VR in gaming

Approximately, 30,000 patents/publications were analyzed for this article. As a simple and quick case study, let’s look at some of the most important and influential patents in this domain to get a glimpse of the emerging technology areas. Our selection is based on (1) total number of forward citations or (2) average forward citations/year. To identify the same, a scatter plot was generated with the estimated remaining life on the X-axis and the forward citations (excluding self-citations) for the active granted patents plotted onto a customizable quadrant, as shown in Figure 3.

Figure 3: Scatter plot of remaining life v. forward citations plotted based on cumulative values (left) or average citations/year (right) and sectioned into Q1-Q4 quadrants

Several patents were identified under Q2 which represent the group of patents with the highest potential to further influence the overall domain. These patents are already generating more than 70 non-self-citing citations per year, since the first publication, with more than 10 years of life remaining till expiry. Additionally, several patents can be identified in the Q3 quadrant that have the potential to move to Q2 in the near future as the technology matures and the patents are cited more by others. The patents in Q3 generate at least 5 citations per year, on average, which can be one of the parameters to determine continued interest in AR/VR technologies in the gaming domain.

US8964298B2 with approximately 143 non-self citing citations per year, since its first publication, and still has 10 years of remaining life before expiry. This patent is titled “Video Display Modification Based on Sensor Input for a See-Through Near-to-Eye Display” and is assigned to Microsoft. The patent discloses a dynamically adjustable display system for an AR/VR HMD that displays a virtual object on the display portion of the HMD with a first set of display parameters (e.g. brightness, color saturation, color balance, color hue, video resolution, transparency, etc.). Upon detecting a change in environmental conditions, using an environmental sensor, the display automatically displays the virtual object with modified display parameters to suit the detected environmental conditions.

Figure 4: US8964298B2 assigned to Microsoft

US9128281B2 with approximately 79 citations per year since its first publication and 12 years of remaining life before expiry is assigned to Microsoft. The patent titled “Eyepiece with Uniformly Illuminated Reflective Display” focuses on the overall hardware design of an HMD with a specific focus on the display assembly. The invention discloses an image source that includes an LED, a planar illumination facility, and a reflective display attached to the mechanical frame of the HMD. The planar illumination facility guides the light from the light source to be projected on to the reflective display, which is further directed towards a beam splitter. The beam splitter then reflects a portion of the image light onto a partially transmissive mirrored surface forming a virtual object.

Figure 5: US9128281B2 assigned to Microsoft

US8957835B2, assigned to Apple, with approximately 10 years of remaining life before expiry is generating an average of 72 citations per year. This patent is focused on an HMD frame that can receive a mobile device to be used as a display component. The frame has a cavity for physically receiving the mobile phone and has a detection mechanism to detect the mobile phone upon mounting, and alter the screen to display images in a manner suitable for an HMD display.

Figure 6: US8957835B2 assigned to Apple

US10203762B2, with approximately 14 years of remaining life, seems to be generating an average of 118 citations per year since its first publication. This patent is assigned to Magic Leap and is titled “Methods and Systems for Creating Virtual and Augmented Reality”, discloses almost all aspects of an AR/VR system including the hardware and software aspects required for an immersive experience. However, the claimed subject matter is more focused on recognizing user gestures using onboard image sensors to enable a user to interact with virtual objects in an AR/VR environment.

Figure 7: US10203762B2 assigned to Magic Leap

US9384594B2, assigned to Qualcomm, generates 84 citations, on average, per year since its first publication and has 14 years of remaining life before expiry. This patent titled “Anchoring Virtual Images to Real World Surfaces in Augmented Reality Systems”, focuses on anchoring virtual objects to user-selected physical surfaces in their environment. The invention uses a wearable camera and sensor array to capture and recognize the user’s environment, detect user input for selecting a first region, and anchor a virtual object to the user-selected region. Further, upon the user selecting a second region, the system calculates the orientation and distance with respect to the wearable camera and moves the virtual object to the second region.

Figure 8: US9384594B2 assigned to Qualcomm

US7227526B2, assigned to Qualcomm, is from the Q1 quadrant of the total citation count scatter plot. It has 2,013 total citations, but has only 2 years of useful life. This patent titled “Video-Based Image Control System” is focused on using stereo cameras to generate a depth field and further detect objects and gestures within its field of view. These detected objects and gestures are then used for interacting with the Virtual and Augmented Reality applications.

Figure 8: US7227526B2 assigned to Qualcomm

According to a recent forecast, AR/VR applications are expected to generate $67 Billion in revenue by 2024 (Source: link). AR/VR has almost become synonymous with gaming. It is increasingly seen as a means for immersive experiences and social connection. This trend will likely continue as we emerge from the pandemic. The AR/VR gaming value chain includes companies and startups working in software, hardware, content development and infrastructure. Beyond gaming, AR/VR also has other useful applications, such as healthcare, engineering, education, entertainment, etc. Companies need to monitor the area broadly for emerging trends and competitive threats from adjacent areas.

Each of the patents illustrated above discusses a different aspect required for providing an immersive experience. Microsoft’s patents focus on the illuminator design on the HMD along with dynamically adjusting the display properties of the virtual object being displayed for an improved visual experience. Magic Leap’s patent focuses on detecting gestures for user input. Qualcomm’s patent focuses on detecting the environment and anchoring virtual objects to environmental surfaces. Finally, Apple’s patent focuses on a modular structure that can use a mobile device as its primary display. MaxVal’s experts can further analyze the AR/VR gaming space and categorize the art into a taxonomy of different technology and application nodes for a better in-depth understanding of the domain, which can be used to generate competitive insights.