You may have heard of OpenGL and Direct3D, but do you actually know what they are? Although oft mentioned in passing with respect to virtual reality, few actually realize how essential they really are. Let me explain.
Just as English and Mandarin are the languages of human communication, OpenGL and Direct3D are the languages of computer graphics. Their purpose is to render graphics, and they’re used to facilitate communication between application developers and hardware driver. Before these standards were created, programmers would have to write different modules for each type of hardware that a user might have. But with a language standard shared among application developers and hardware companies, programmers no longer have to account for different types of hardware. Instead, the GPU drivers are now responsible for understanding the OpenGL/DirectX code and executing them on different low level architectures. See the figure below for a simple view of how OpenGL games are processed in a Linux system.
How do these two graphics APIs relate to VR? Basically, VR displays are no different from other graphical displays. The only difference is in their presentation, which uses stereoscopic tricks in order to convince us that we’re in an immersive virtual space. If you’ve ever developed VR software in Unity (a game development engine), the hard work has been abstracted away from you. Under the hood, however, Unity still generates OpenGL or Direct3D code to run on your head mounted display (HMD). See below for an example of how a VR headset renders Team Fortress.
By the way, if you’ve also heard of OpenVR, that’s slightly different. OpenVR is more of an open API designed to develop logic that can interact with all aspects of a VR device. This includes not only the HMD, but also the controllers, cameras, sensors, etc. We will cover this and the Oculus SDK in more detail in an upcoming blog post.
What’s the Difference?
OpenGL stands for Open Graphics Library and, as its name implies, is designed to be ‘open’. It has implementations available for nearly all operating systems including Microsoft Windows, Linux, MacOS, Android, IOS and game consoles developed by Nintendo and Sony. Besides Windows, most operating systems that support hardware-accelerated 3D graphics have chosen OpenGL to be their primary graphics API. The standards of OpenGL are maintained by a consortium composed of companies from several areas to ensure a wide compatibility. Yet this compatibility comes at a cost. OpenGL is fairly slow at accommodating new technologies, due to the different companies’ concerns about compatibility.
In contrast, Direct 3D is a member of DirectX, a collection of APIs specially designed for Windows operating systems, including the OS that runs in the Xbox. Because it relies on other Windows components, it only supports Windows (although some hackers might be able to run a limited portion of Direct 3D on Linux by using Wine, a compatibility software layer). Ignoring compatibility concerns enables Direct3D to innovate much faster than OpenGL. This has become increasingly important: Windows and Direct3D currently dominate PC gaming.
The PlayStation 4 is one of the rare devices that supports Direct3D. Due to the specific hardware differences between the architecture of the PS3 and the PC’s X86, the PS systems once only supported a modified version of OpenGL. Eventually, however, Sony switched the PS4 to the X86 architecture in order to reduce cost and simplify the game development process. Consequently, the PS4 now supports a modified version of DirectX 11. However, the forms of OpenGL and DirectX used on PlayStations are still not original versions. They are additionally empowered by lower level access to core system hardwares such as the GPU, CPU and RAM. Theoretically, this means PS4 DirectX/OpenGL will be more powerful than the original versions.
OpenGL was one of the first graphics languages created. It was initially developed by SGI (Silicon Graphics Inc.) in 1992, and was used primarily on professional workstations. Shortly after, however, SGI saw it’s potential for use on a broader scale. They opened the API and established the OpenGL Architecture Review Board, a consortium including Apple, IBM and other major players. In 2006, ownership was transferred to Khronos Group, another consortium aimed at developing open APIs.
Three years later in 1995, Microsoft launched its own Graphics API, Direct3D. Because it was optimized for Windows operating systems, Direct3D is more efficient and has more functionality than OpenGL on Microsoft machines. As a result, Windows more or less dominated the PC game market in the 2000s. While some gaming consoles like PlayStation did support OpenGL, instead of developing it further these consoles concentrated on developing their own customized API extensions.
In 1997, Windows and SGI proposed a plan to merge OpenGL and Direct3D. Unfortunately the project was eventually canceled in 1999, due to a variety of disagreements. But the two APIs are not entirely incompatible: Valve, the company that owns Steam, developed a wrapper to translate Direct3D applications into OpenGL, to allow Dota2 to run on Mac and Linux.
Recently, however, OpenGL has steadily regained market share as more and more people switch to Macs and the smartphone gaming market booms. For example, many VR applications are developed for cell phones running IOS and Android and thus use OpenGL (sadly windows phone only have about 1% of market share). It appears that in today’s environment, OpenGL’s versatility is proving increasingly valuable.
Future and VR in Web
Although nearly all platforms support the OpenGL API, developers still need to create individualized applications for different platforms. This is because graphics are only one part of an app, and other components still differ between platforms. To remedy this, WebApp was developed to run entire programs in a web browser such as Chrome and Firefox, so that one program can run on all platforms with a compatible browser. Meanwhile, on the client side everything is transmitted to on the fly from servers, so no further app installation is required. A special API called WebGL has also been created for the Web, and is based on OpenGL ES, a subset of OpenGL developed specifically for mobile devices. To accomplish a similar goal for VR, another API called WebVR has been developed to easily bring VR experiences to browsers, no matter the platform.
WebGL and WebVR have not yet reached full maturity. They still face the bottleneck of internet bandwidth limitations, and have some drawbacks in terms of efficiency. Currently, the only two browsers to officially support WebVR are Firefox Nightly with a WebVR extension and a special build of Chrome. Nonetheless, some companies are experiencing serious breakthroughs on WebVR. For example, Mozilla is developing a framework called A Frame to create WebVR contents using markup languages like HTML. Given the enormous potential benefits of easy-to-use, platform-agnostic standards, as well the quality work being done to build out these APIs, I am confident we’ll be seeing more and more of WebGL and WebVR in the recent future.
Hengte Lin, Computational Science and Engineering Master student at Harvard
Zhifan Wang, Translation and Interpretation Master student at Middlebury Institute of International Studies at Monterey
Evan Brown, Harvard College
OpenGL, Wikipedia, https://en.wikipedia.org/wiki/OpenGL
Visualization-of-game-genre-and-platforms-since-1975, Daystodev, http://www.daystodev.com/visualization-of-game-genre-and-platforms-since-1975/
Valve’s Direct3D to OpenGL wrapper on Github, https://github.com/ValveSoftware/ToGL
Team Fortress VR mode, https://www.youtube.com/watch?v=XD7euJtTQio
OpenGL website, https://www.opengl.org/
Direct3D documentation, https://msdn.microsoft.com/en-us/library/windows/desktop/hh309466(v=vs.85).aspx
WebGL website, https://www.khronos.org/webgl/
WebVR website, https://webvr.info/