n this blog post, I’ll focus on how the low-latency capabilities of 5G are enabling a new class of experiences at scale, namely cloud gaming, boundless VR, and boundless AR.

5G enables low-latency applications at scale

5G was designed from the ground up for low-latency and high reliability. For example, the 5G NR scalable numerology allows for shorter slot times, especially in the mmWave and mid-band spectrum, which reduces wait times. 5G also provides faster processing timelines thanks to techniques like front-loaded reference signals, faster acknowledgements, faster grant to uplink transmission, and fast channel quality indicator (CQI) feedback based on enhanced reference signals.  

5G NR provides the framework for fast, reliable beamforming and beam tracking that can deliver improved throughputs and low latency communication even with mobility and non-line-of sight (NLOS) scenarios. Advanced features like mini slots can enable ultra-reliable low latency short data transmissions, such as uplink pose to start at any time.

5G is built for low-latency and high reliability.

5G’s low latency, high capacity, and high reliability has opened the door to a new era of distributed computing, where workloads can be distributed between the device and cloud to provide new capabilities and enhanced experiences.

Optimizing 5G to achieve boundless XR KPIs

We’ve been working on boundless XR for the past five years. For context, boundless XR distributes processing through split rendering to deliver truly immersive XR experiences. On-device rendering and perception processing is augmented by high-performance edge-cloud graphics rendering over a high-capacity, low-latency 5G connection. As you can imagine, making the XR experience truly immersive requires a system solution across the device, edge cloud, and 5G network that guarantees low latency and high reliability. That’s where Qualcomm comes in — end-to-end 5G optimizations are our bread and butter.

Our multi-user 5G boundless VR system uses commercial products and platforms.

In May of 2020, we kicked off 5G mmWave over-the-air trials of boundless XR running on commercial equipment and platforms at both Qualcomm and Ericsson campuses, initially targeting VR applications for private enterprise networks. We’ve made significant progress since then and demonstrated at MWC 2021 how our 5G end-to-end optimizations allow us to achieve the initial key performance indicators (KPIs) for at-scale 5G boundless VR deployments.

A KPI for boundless VR is motion-to-render-to-photon (M2R2P) latency, which is the round-trip time from a user’s motion, to the corresponding visual being rendered in the edge cloud, and finally to the image being displayed on the VR headset. The 5G round-trip time is a key component of M2R2P latency, and around 20 ms is a reasonable target for good user experience. Large 5G round-trip latencies can cause user experience degradation. For example, with a 90 frame per second VR display, having 1 percent of frames with large latencies leads to a poor experience approximately every 1 second, so it’s essential to minimize the occurrence of high latency.

In real-life deployments, RF and interference changes happen frequently, which can cause performance issues and latency spikes. In our testbed, we replicated two challenging scenarios that caused a sudden spike in 5G round-trip time for the baseline 5G mmWave system: a blockage scenario in which a VR user is blocked from the gNodeB by another user and a mobility scenario in which the VR user rotates his head 180 degrees. In both cases, our 5G optimizations help minimize the latency spike, providing a seamless user experience.

5G optimizations can be made both on the device and the network to improve the user experience. 5G device optimizations include low-latency packet processing, adaptive beam management for improved reliability, and dynamic user plane protocol adaptation to ensure that the application receives essential packets of a frame in time.  In fact, we achieved a >25 percent reduction in tail latencies with device optimizations alone.

5G network optimizations, such as gNodeB deployment, network parameters, and a quality-of-service and delay-aware scheduler, can also have substantial impacts on latency. Besides improving the user experience, end-to-end optimizations translate to capacity improvements. This is important since latency reduction can be traded off for relaxed packet delay bounds, leading to higher system capacity. Our over-the-air (OTA) mmWave system can support 6 VR users (each with 50Mbps of DL traffic) per gNodeB on 100MHz of system bandwidth.  With further optimizations, such as techniques enabling multiple users to transmit uplink data in the same uplink slot, our system can be scaled to support more than 12 boundless VR users per gNodeB.  A private network deployment, such as in enterprise, industrial, or arena gaming, may deploy multiple gNodeBs to improve coverage and support a large number of users.

In private networks, boundless XR is ready for deployment across a variety of use cases.

Optimizing 5G for boundless AR

The AR device of the future will be sleek, stylish, and light-weight glasses that provide immersive digital augmentation to enhance our lives. This is where we want to go, but how do we get there? The industry has a clear focus on low-power, low-latency 5G connectivity, device processing, and sensors to make this a reality. For the connectivity part, there is a strong roadmap of 5G NR features that will further enhance device power efficiency with advanced power saving techniques. For example, Release 15 includes bandwidth part (BWP), which allows for narrowing the bandwidth when there is low traffic arrival. Release 16 enables cross-slot scheduling, which increases the gap between control and data to increase modem sleep. Release 17 includes discontinuous physical downlink control channel (PDCCH) monitoring, which allows for faster transition to sleep after an XR traffic burst. Release 17/18 features in discussion include enhanced connected mode discontinuous reception (eCDRX), which aligns the awake cycles of the radio frequency (RF) reception to the video frame rate, which allows the modem to have more continuous sleep time and to reduce power consumption.

5G NR has a strong roadmap for further enhancing device power efficiency for applications like XR.

Most of these techniques will also be useful for VR and other power-constrained devices.

Optimizing 5G to achieve cloud gaming KPIs

5G is creating new experiences with cloud gaming on mobile devices. With reliable, low-latency 5G connectivity, you can have console-based cloud gaming in the palms of your hands virtually anywhere you go. This means having an opportunity to play a broad catalogue of games ranging from AAA blockbusters to popular cross-platform titles. Ensuring a low-latency 5G connection allows for faster input reaction times from the device back to the cloud, as well as simultaneous multi-player interactions.

Just like M2R2P latency is important for XR, controller-to-render-to-photon (C2R2P) latency is crucial for cloud gaming. For a smooth gaming experience, C2R2P latency should be kept below 133 ms (based on Gamebench latency ratings).  Given processing times required at the cloud/edge, and on-device processing times including controllers, it is good to keep 5G round-trip time below 50 ms.  In our measurements, we observed that cell edge and mobility scenarios due to dynamic RF environment can cause large latencies. Since the cloud gaming system architecture utilizes similar building blocks as boundless XR, many of the same 5G optimizations can be applied for low latency.  With our 5G device optimizations alone, we achieved 35 percent reduction in tail latencies.

5G device optimizations reduce the latency and improve the user experience

From a network standpoint, migration to 5G standalone (SA) mode will help to further improve latency since the 5G Next Gen Core is optimized for latency and the 5G RAN handles both the data and control from the device. The migration from central cloud to local edge cloud will also improve latency.

Additionally, upcoming 5G NR releases will provide enhanced mobile broadband (eMBB) improvements that will further improve coverage, capacity, latency, and mobility. These features will improve boundless XR and cloud gaming user experience.  For example, Release 16 has features like enhanced downlink/uplink MIMO, more robust mobility with minimal interruption during handover, and enhanced ultra-reliable low-latency communications (eURLLC) with uplink power boosting and cancellation. Release 17 has further improved MIMO for higher mobility, enhanced mobility for mixed topologies, and other features like >4 receive antennas, 1024-QAM, and multi-SIM.

This is just the start. 5G is a platform for innovation and will introduce many new experiences. We’re excited to see our end-to-end 5G optimizations enable the mass deployment of low-latency experiences, like boundless XR and cloud gaming, as well as whatever comes next.