Introduction
One of VR’s greatest hurdles is video quality (resolution). From the Google Cardboard to the HTC Vive, 360-degree videos are frequently plagued by choppy frame rates and pixelated images. Previously, it seemed that the answer would be 4K Ultra HD (UHD) cameras that recorded in 360-degrees. After all, don’t 4K videos look brilliant on Netflix and YouTube?
However, despite 4K UHD being heralded as a next-generation recording format, it’s a common misconception that it yields similar results for 2D and 3D. This blog will explain the current limitations in viewing, recording, and streaming technology, as well as the developments that will finally put the ‘reality’ in Virtual Reality.
The lowdown: VR video resolution is limited by the current state of hardware technology. for its full potential to be realized, time and innovation will be required in:
- Viewing Hardware: To increase pixel count on screens and eliminate ‘screen door effect’
- Recording Hardware / Software: To record in 12K resolution as 4K is insufficient for VR
- Internet Streaming: To increase internet bandwidth and stream large VR files seamlessly
It’s All About Pixels
Firstly, it’s important to understand that the resolution of any video, 2D or 3D, depends on the number of pixels in your view. Each pixel provides one shade of color, and the more pixels you can compress into a frame image, the more details, and definition you can add. This leads to the distinction between Standard Definition (SD: 240p – 480p) and High Definition (HD: 720p – 4K), with benefits such as higher picture quality:
Or a similar quality at a larger size:
However, the way VR is currently being recorded and consumed is not as optimal.
Viewing Hardware: ‘Screen Door Effect’
Take the device that you’re viewing this on, and move the screen right in front of your eye. Do you see an endless grid of tiny squares? Those are individual pixels, and the closer you look, the more you will notice the blank spaces between each pixel.
These spaces become more obvious the closer you get to a screen, and your eye subconsciously looks past them to the focus on colored pixels. This creates a blurring effect, like looking through a screen door (hence the name):
You don’t notice this much in daily life because you view smartphones as the manufacturer intended, about 20-30cm away from your eyes. When you bring a screen as close as 5cm to your eyes, and then magnify them with VR lenses, you are seeing a lot fewer pixels in your field of vision.
Solution: VR headsets with higher pixel density, measured by pixels per inch (PPI; dots per inch, or dpi, is a similar term). The Samsung S7’s screen currently sits at 577 pixels per inch. In the future, it plans to release standalone headsets with in-build screens (no more smudgy phone screens!) that provide 1,200-1,500 PPI. The PSVR also incorporates 3 sub-pixels in each pixel to minimize the screen door effect.
Recording Hardware / Software: Limitations of 4K Video
Despite the ‘4K’ claim, the average 4K Ultra HD video has a resolution of 3840 x 2160 pixels. Using a bit of math and a GearVR, we can better understand why this isn’t enough yet.
In short, the hardware and software now can’t deliver 360 videos in the 4k quality a layperson expects. If you really really wonder why brace yourself for some math ahead!
The resolution of the Samsung S8 is 2960 x 1440. We must split this in half lengthways (VR video displays one of the same videos for each eye) and account for lens distortion, giving us an effective screen resolution of 1480 x 1280, allowing your eye to take in 1,894,400 pixels.
The GearVR provides a 90-degree field of view (FOV), so at any time, you are only looking at 1/4 of the 360-degree horizontal view, and 1/2 of the 180-degree vertical view. When you use this information to calculate the average number of video pixels in the display FOV (Horizontal = 3840 ¸ 4, Vertical = 2160 ¸ 2), you are left with an effective video resolution of 960 x 1080, which is 1,036,800 pixels.
Given that the headset puts 1,894,400 pixels in your eye’s FOV, the image stretches across a wider surface area, providing 54.7% of the original resolution. Alternatively, imagine taking an image from a screen, and wrapping it around your head:
Naturally, loss of image quality will occur.
Solution: While 4K is currently the de-facto HD recording format, it simply is not enough for VR. This is especially difficult since 4K is already very memory-intensive, with a 10-minute video taking up 1GB of space on average. Technology will have to improve to allow for 8K and even up to 12K to be affordable and memory-efficient, which is not the case for now. (Fun Fact: The human eye is estimated to have a 576-megapixel or 16K resolution!)
Internet Streaming: Bandwidth Limitations
Internet bandwidth has been a problem since the creation of online video; if you can’t download fast enough, you can’t view it well enough. Anyone who’s ever used YouTube and Netflix is all too familiar with the sudden blurring of their video, ruining their viewing experience for only a moment (or thirty). For VR, this is especially noticeable as it disrupts the feeling of immersion, a core element of the medium.
Furthermore, the large size of VR files make single downloads unfeasible; streaming will continue to be the main channel of data delivery. However, even Netflix’s non-VR 4K videos are said to work only with a 25mbps connection, while the requirement for fluid 4K VR videos tips the scales at a whopping 500mbps.
Solution: This is a waiting game as well, but hopefully not for long. Nielsen estimates that internet bandwidth is constantly increasing at 50% per year, which would increase the population that has ready access to high-speed internet. Companies are also developing innovative streaming protocols to accommodate a large amount of data throughput, such as Visbit, which uses their patented View-Optimised streaming to only provide foveated high resolution streaming in places where the viewer looks, minimizing bandwidth used.
Given that the headset puts 1,894,400 pixels in your eye’s FOV, the image stretches across a wider surface area, providing 54.7% of the original resolution. Alternatively, imagine taking an image from a screen, and wrapping it around your head:
Going Forward:
It is now more apparent that the current state of technology is not fully prepared to deliver flawless VR content. The level of immersion required is high because it is an all-or-nothing experience: You are either totally blown away, or you are not. However, the current state of innovation is what pushes the frontier of the industry, and provides exponential growth. What we see now is but a glimpse of the true potential of VR, and with a global effort in full force, we may just realize it soon enough.
