Innovations in Unreal Engine

Why is Unreal Engine key to VR development?

In recent years, Unreal Engine has become a competitive tool for game developers and creators of VR content. Continuous advancements and new features brought opportunities and challenges that developers must keep up with. Below, you can read our detailed overview of the most important innovations in the latest versions of Unreal Engine.

Unreal Engine 5.3: Innovations in VR development

The Unreal Engine 5.3 version introduced numerous updates, several of which directly impact the work of VR developers:

• Nanite and Lumen improvements: These technologies provide even more detailed graphics and realistic lighting, which are crucial in VR applications.
• Full support for Pico headsets: This version offers full compatibility with Pico headsets, an essential feature for developers working with this platform.
• Advanced VR motion mechanics: Simplified interactions also improved the efficiency of VR development.

Unreal Engine 5.4: The era of animation and Meta headsets

The 5.4 version brought a range of improvements that position Unreal Engine at the forefront of dynamic motion and animation:

• Animation Warping: This feature enables motion animations to dynamically adapt to different scenarios, delivering a much more realistic VR experience.
• Enhanced support for Meta headsets: Version 5.4 introduced significant improvements in the stability and functionality of Meta headsets, providing considerable assistance to developers.
• Expanded VR APIs: New toolsets and APIs offer even more opportunities for developing VR applications.

Unreal Engine 5.5: A revolution in graphics and compatibility

Version 5.5 revolutionized the possibilities of VR development, especially in terms of graphical rendering and plugin management:

• Nanite foliage support: Highly detailed vegetation now allows the creation of even more realistic VR environments.
• Enhanced support for OpenXR plugins: OpenXR-based development became even more seamless, particularly when integrating specific functions.
• DirectX 12 priority: Lumen technology is now exclusively available under DirectX 12, which delivers better graphical results but requires compromises in terms of stability.

Rendering solutions: Forward vs. Deferred Shading

The rapid advancement of VR technology presents new challenges for developers, particularly regarding efficient rendering methods. Unreal Engine's two primary rendering techniques, Forward Shading and Deferred Shading, each have their advantages and disadvantages, making it essential to understand these differences before choosing either. Let’s take a closer look at how they differ and which might be the best choice for your VR project.

Forward Shading: Simple, fast és stable

Forward Shading is one of the more traditional rendering methods and is popular in VR development due to its numerous advantages. The simpler structure delivers excellent performance, even on lower-end hardware, which is especially important if the goal is to provide the most stable experience possible.

This technique is ideal for those prioritizing performance and stability. It requires less VRAM, so even lower-end graphics cards can run the application at adequate speed. Additionally, it operates more reliably under DirectX 11, reducing the likelihood of errors and stuttering.

However, like any solution, Forward Shading has its limitations. It does not support advanced visual features like dynamic global illumination or Lumen, which means compromises must be made in terms of realism. If your project places a strong emphasis on detailed lighting and shadow effects, this method may not meet your expectations.

For projects based on Unreal Engine's default VR Template, Forward Shading is set as the standard rendering solution.

Deferred Shading: Visuals above all

If visual quality is the top priority, Deferred Shading is the perfect choice. This advanced technology enables dynamic global illumination and, with Lumen integration, offers stunningly realistic lighting and shadow effects.

Deferred Shading is particularly powerful when working with complex geometry and detailed textures. Combined with Nanite technology, it can seamlessly handle models with extremely high polygon counts without compromising on detail.

However, the impressive visuals come with higher hardware requirements. Deferred Shading places a significant load on systems, especially in terms of VRAM and CPU usage, making it recommended only for higher-end configurations. Additionally, when using DirectX 12, you may encounter compatibility and stability challenges that require careful optimization.

Which one should you choose?

Selecting the right rendering technique depends primarily on your project’s goals and the available resources.

If speed and stability are your top priorities—such as for VR applications with lower hardware requirements—Forward Shading is the ideal solution. However, if visual quality is a key focus and you have the necessary hardware, the combination of Deferred Shading and Lumen can deliver unmatched results.

It's also crucial to consider whether your project will run as a standalone application—entirely on the VR device—or as a PC VR application. In the latter case, your computer’s performance will likely be able to handle more demanding tasks than the VR headset alone.

Remember, the goal is to choose the technology that best suits your project’s needs and leverages the full potential of Unreal Engine.

The Role of OpenXR: Why has it become a standard in the VR/AR World?

In recent years, virtual reality (VR) and augmented reality (AR) technologies have gained immense popularity, with countless devices entering the market. However, this growth hasn’t been without challenges: for a long time, developers have had to use unique SDKs (Software Development Kits) for different devices, significantly complicating their work. OpenXR offers a solution to this problem.

What is OpenXR and why is it important?

OpenXR is an open standard developed by the Khronos Group, designed to provide a unified interface for supporting VR and AR devices. OpenXR allows developers to work with a single API compatible with almost every major VR/AR platform. This is a huge advantage, as it eliminates the need for developers to rely on the proprietary SDKs of various hardware manufacturers.

Why is everyone using OpenXR?

OpenXR has rapidly become the standard among VR/AR developers, and for good reason. Here are the key benefits:

• Unified API: With OpenXR, developers can work with the same code regardless of the device they’re developing for. This not only saves time but also reduces the likelihood of errors.
• Platform independence: Major hardware manufacturers like Meta, Pico, and Varjo fully support OpenXR. This means an OpenXR-based application is almost guaranteed to work seamlessly on most devices.
• Native OpenXR support in Unreal Engine: Unreal Engine, one of the most popular game engines, offers built-in support for OpenXR. This allows developers to implement core functionalities across all devices effortlessly.

OpenXR plugins: Expanding functionality

While OpenXR provides basic support for most devices, advanced features require additional plugins. Unreal Engine comes with native OpenXR support and when we use the VR Template project, this add-on is automatically enabled.

This setup gives developers a manufacturer-independent framework with the core VR development functionalities. It’s a powerful tool because it eliminates the need to handle specific hardware compatibility. With this, retrieving data from controllers and headsets becomes seamless, and even basic eye-tracking functionalities are supported. Developers can access various data such as acceleration and gyroscope readings effortlessly.

The exciting part begins when developers want to leverage device-specific features like advanced eye-tracking or face-tracking. To utilize these specialized capabilities, plugins from hardware manufacturers (like Pico, Meta, and Varjo) are required. These plugins unlock the full potential of the devices, enabling cutting-edge functionality within Unreal Engine.

And with this, we arrive at the most critical part of this article.

Why is version control crucial in VR Development with Unreal Engine?

In VR development, choosing the right version of Unreal Engine fundamentally determines the success of the project. Since new versions always offer exciting, cutting-edge features, it can be tempting to upgrade. However, these new opportunities often come with risks, such as compatibility issues, instability, or challenges arising from the need to update third-party plugins. But why is this such a challenge, and how can we address these problems?

Downgrade: More problems than solutions

Reverting to an earlier version, known as a downgrade, is often a forced decision, but the issues it brings frequently cause more harm than good. One of the most common problems is changes in data structures. New versions often reformat project files, leading to incompatibility with older versions and potential data loss.

Additionally, the continuous updating and removal of features can create confusion. Epic Games periodically modifies the available toolset, meaning older tools may become obsolete over time, requiring rework that can significantly slow development. Downgrading can also introduce hidden bugs, which may increase future workload and costs.

Challenges with third-party plugins

Plugins are essential in VR development, as they accelerate workflows and offer numerous possibilities. However, they can become a source of problems during version upgrades. Since not all plugins are compatible with different versions of Unreal Engine, it’s crucial to verify whether the plugins you use support the new version before updating.

API changes can also pose challenges, as they may impact existing code and require significant adjustments. Another common issue is that plugin developers may stop support for the plugin, which can create serious difficulties for long-term projects.

How to choose the right version?

The key to successful VR development lies in deliberately selecting the appropriate version of Unreal Engine while considering the project's specific needs. The following tips can help you make the best decision:

• Choose a stable version for long-term projects: For longer-term projects, it’s wise to opt for a proven, stable version to minimize the risks of instability.
• Check plugin compatibility: Always ensure that the plugins used are compatible with your chosen version to avoid unexpected issues.
• Upgrade only when necessary: New versions offer exciting features, but you should only adopt them if they provide significant advantages for your project.
• Test with a prototype project: Before updating an entire project, create a test project in the new version. This allows for detecting errors early and preventing larger issues.

What do the latest versions of Unreal Engine offer??

Unreal Engine is constantly evolving, and while every version introduces new features, not all are ideal for VR projects. Here are some highlights of recent updates:

• Unreal Engine 5.3: At the time of publishing this article, this is the latest version supporting Pico VR headsets. It also introduced significant improvements in Nanite and Lumen technologies.
• Unreal Engine 5.4: This version focuses on Meta VR headset support but is no longer compatible with Pico headsets. New features include Animation Warping and enhanced VR support.
• Unreal Engine 5.5: Comes with Nanite foliage support and improved visual quality, but compatibility with VR headsets varies.

To ensure your project's success, it’s crucial to carefully evaluate when and why you upgrade versions, keeping hardware and software compatibility in mind. Additionally, consider whether the plugins you rely on are essential. As mentioned before, OpenXR provides the functionality to accomplish almost everything across devices, except for specific advanced features.

So, one of the key questions to ask yourself is: Do I need Face Tracking or Eye Tracking?

Hardware and Compatibility: What to watch out for?

The hardware requirements of VR development often dictate which Unreal Engine version is worth using. Here are some examples:

• Meta VR headsets: Supported up to version 5.4. The Movement Sample project available on GitHub can serve as a helpful starting point.
• Pico VR headsets: Only compatible with version 5.3, so using newer versions with these headsets is not recommended.
• Varjo VR headsets: Compatible with version 5.4, but their high hardware requirements can be a limiting factor.

Understanding the compatibility of your chosen hardware with Unreal Engine versions is critical for avoiding unnecessary roadblocks during development.

Hardware Compatibility and plugins in Unreal Engine VR development

The success of VR development largely depends on how well we choose the target hardware and the software versions that match them. Support for different VR headsets can vary significantly across Unreal Engine versions, directly impacting project stability and functionality. Let’s explore how to approach this crucial aspect in detail.

Meta VR Headsets

Meta's VR headsets are particularly popular among developers due to their stability and the wide range of available accessories. They are supported up to Unreal Engine 5.4 and have proven exceptionally reliable for DirectX 11-based applications. For beginner developers, the Meta-provided Movement Sample project, available on GitHub, can be an excellent starting point. It offers ready-made VR movement mechanics, making the initial steps significantly easier.

Pico VR Headsets

Support for Pico headsets is somewhat different. Unreal Engine 5.3 is the last officially supported version—currently—but the Pico OpenXR plugin offers partial compatibility with Unreal Engine 5.5. While this comes with limited functionality, it still provides a way to leverage the latest technologies. It is important to note that the Pico XR plugin works exclusively with Unreal Engine 5.3, so this should be considered before starting development.

Varjo VR Headsets

Varjo headsets represent the pinnacle of the industry, but this comes at the cost of significant hardware demands—and expensive devices. They are fully compatible with Unreal Engine 5.4 and enrich VR development possibilities with new features. If you're aiming to deliver a high-end VR experience, Varjo devices are an excellent choice. However, it is crucial to carefully consider the system requirements in advance.

Choosing the right Unreal Engine version

Selecting the appropriate engine version is crucial in VR development. This decision impacts not only compatibility but also the efficiency and stability of the project. The following factors should be considered:

• Hardware requirement: Always check whether the target device is compatible with the specific version of Unreal Engine. This helps prevent compatibility issues down the line.
• Testing plugins: It's a good practice to test third-party plugins early on to ensure they work properly with the chosen version.
• Focus on stability: For long-term projects, it's recommended to choose a stable, well-documented version of Unreal Engine to ensure a smooth and consistent development process.

Technical limitations and challenges

DirectX version support

In Unreal Engine VR development, choosing the right DirectX version is crucial. DirectX 11 remains popular due to its stability, especially in VR projects. However, switching to DirectX 12 opens up the possibility of using Lumen, although this often comes at the cost of increased stability issues.

Asset compatibility

The assets used for VR projects are often only compatible starting from Unreal Engine 5.4. Therefore, it's best to review the version requirements of the assets before beginning development to ensure smooth integration.

Nvidia DLSS: Is it necessary for VR development?

Nvidia DLSS (Deep Learning Super Sampling) technology has revolutionized game development, particularly in projects where balancing graphic performance and visual quality is the goal. In VR development, where every frame is crucial for a smooth experience, DLSS can play a key role. But is it really necessary?

DLSS uses artificial intelligence to increase the resolution of images without compromising graphical quality. It achieves this by rendering a lower-resolution image and then enhancing it using AI, ensuring a higher resolution and more detailed visual experience while significantly reducing hardware load.

Why is DLSS important in VR development?

In VR, low latency and a consistent frame rate are crucial, as the user experience directly depends on how smooth the visual presentation is. DLSS allows even weaker systems to deliver a detailed visual experience without compromising performance. For example, if a project aims to deliver a 4K resolution experience in VR, DLSS can help the system render at the lower 1080p resolution and then use AI to upscale it, reducing the load on the GPU.

When could DLSS become necessary?

If your VR project aims for cutting-edge graphics and detail, but not everyone has high-end hardware, DLSS technology could be the solution. It can be especially useful when dealing with high-resolution textures, complex geometry, and intricate lighting effects in VR development. With AI-driven upscaling, users can enjoy a smooth, crisp visual experience even on mid-range or lower-tier systems, while lowering the required frame-per-second (FPS) value, significantly reducing system load.

What are the drawbacks of using DLSS?

While DLSS offers numerous advantages, it is not without its drawbacks. Since the technology relies on artificial intelligence, certain types of graphical elements might not be perfectly rendered, particularly in situations where fine details and subtle nuances are crucial. Additionally, DLSS only works with Nvidia RTX graphics cards, so users without such hardware will not have access to the technology.

Another issue is version compatibility—currently, DLSS is only supported up to Unreal Engine 5.4.

DLSS and VR: How to use it effectively?

If you do decide to integrate DLSS into your VR project, it's essential to test and optimize different settings to find the right balance between visual quality and performance. The biggest benefit of DLSS is that by saving system resources, it enables you to incorporate visual effects that the system would otherwise struggle to handle, all while ensuring a smooth user experience.

Experiences and Recommendations

For successful VR development, it’s important to follow a few best practices:

• Selecting the Development Environment: Choose an Unreal Engine version that is stable and compatible with all target hardware. This ensures smooth progress and avoids compatibility issues.
• Managing Updates: Avoid too frequent version changes, as they can increase the likelihood of bugs and extend development time. Make updates only when necessary to incorporate essential improvements or features.
• Regular Testing: Test the project on all target platforms to ensure a stable and high-quality user experience. Early testing can catch issues and allow for adjustments to improve the final product.

Summary

One of the biggest challenges in Unreal Engine-based VR development is managing compatibility issues between constantly evolving engine versions and hardware. The OpenXR standard is a great help in this regard, as it provides a unified platform for developers. To achieve the best possible outcome, it’s crucial to carefully consider which Unreal Engine version to select and which hardware to target during development.

Sources

• Unreal Engine Official Documentation: https://www.unrealengine.com
• OpenXR Official Documentation: https://www.khronos.org/openxr
• Movement Sample GitHub Project: https://github.com/oculus-samples/Unreal-Movement
• Meta, Pico, and Varjo VR Headset Support Pages
• Technical Blogs and Developer Community Resources
• Nvidia DLSS: https://developer.nvidia.com/rtx/dlss/get-started

Article written by F. Tamás, translated by F. Flóra