Measuring WebRTC Call Quality - Part 1

High-quality video conferencing relies on participants being able to see and hear each other clearly, without any disruptions or distortions. The final call quality depends on various factors, including device capabilities and the quality of the network connection between the sender and receiver.

To evaluate the quality of a WebRTC call, it’s crucial to identify key metrics that offer insights into the call’s performance. These metrics reveal what’s happening behind the scenes, allowing us to diagnose and resolve issues effectively. Common problems like choppy audio, video pixelation, and sync issues can be traced by analyzing these metrics. By determining whether the issue originates from the sender’s or receiver’s side, we can take targeted steps to improve the call quality.

In this blog, we’ll explore critical metrics provided by the WebRTC getStats() API that help measure and debug call quality issues. We’ll first focus on the sender side, explaining how to interpret these metrics and offering practical examples to diagnose and improve call quality.

Key Metrics on the Sender Side

To start with, let's examine the metrics that impact the quality on the publisher's side. These metrics are found in the RTCOutboundRtpStreamStats section of WebRTC's RTCStatsReport, which is retrieved using the getStats() API. For the most accurate data, it's recommended to call the getStats() API every second.

Frame Width and Height

frameWidth and frameHeight represent the width and height of the last encoded frame respectively. The video resolution, calculated by multiplying width and height, determines the clarity and detail of the video

• frameWidth: The width of the video frames in pixels.
• frameHeight: The height of the video frames in pixels

Higher resolution (larger frame width and height) generally means better video quality but also requires more bandwidth and processing power.

Purpose of measuring frame width and height

• Diagnosing Quality Issues: If the video quality is poor, checking the frameWidth and frameHeight can help determine if the issue is related to low resolution.
• Monitoring Performance: These metrics can be used to monitor the performance and resource usage of the video stream. For instance, streaming at higher resolutions can increase CPU and network load.
• Adapting to Network Conditions: WebRTC can adapt to changing network conditions by adjusting the resolution. Monitoring these metrics can help understand how the application is adapting to bandwidth availability.

Frame Rate or Frames Per Second (FPS)

framesPerSecond represents the number of video frames being processed per second.

Purpose of measuring framesPerSecond

• Video Smoothness: Higher FPS values result in smoother video playback, which is important for user experience.
• Performance Monitoring: Monitoring FPS can help in diagnosing performance issues, such as dropped frames or video lag.
• Adaptation to Conditions: WebRTC can adjust the frame rate based on network conditions and device capabilities. Monitoring FPS helps understand how the application adapts to these changes.

Bytes Sent

bytesSent metric represents the total number of bytes transmitted over a specific RTP stream, including both the payload (audio or video data) and the RTP packet headers.

Purpose of measuring bytesSent

1. Bandwidth Monitoring: Tracking the bytesSent metric over time provides insights into the bandwidth consumption of the outgoing stream, which is essential for assessing network load and performance.
2. Bitrate Calculation: You can calculate the bitrate (data rate) by measuring the change in bytesSent over time.
3. Troubleshooting: Monitoring bytesSent can help in diagnosing issues related to packet loss, network congestion, or other transmission problems.

Outgoing Bitrate

The upload bitrate or outgoing bitrate represents the rate at which data is transmitted from the local peer to the remote peer, typically measured in bits per second (bps). A stable bitrate is key to ensuring smooth audio/video playback and a consistent user experience.

Purpose of measuring outgoing bitrate

1. Media Quality: Outgoing bitrate directly affects media quality. Higher bitrates generally lead to better video resolution and clarity, as well as improved audio fidelity, ensuring a superior viewing and listening experience.
2. Bandwidth Utilization: Monitoring outgoing bitrate provides insights into the application’s bandwidth usage, helping maintain operations within the network’s capacity and preventing congestion.
3. Identifying Performance Issues: Persistently low outgoing bitrates may indicate network issues, server limitations, or client-side bottlenecks, making it a valuable metric for diagnosing and resolving performance problems.

By tracking outgoing bitrate, developers can optimize network and computational resources, adjusting video resolution, frame rate, or codec settings based on the available bandwidth and current network conditions.

Packets Lost

packetsLost metric represents the cumulative count of RTP (Real-time Transport Protocol) packets that were expected to arrive at the receiver but did not. This count starts from the beginning of the RTP stream and includes all packets lost during the session.

Causes of Packet Loss

1. Network Congestion: High traffic can cause packets to be dropped if network devices cannot handle the volume.
2. Network Errors: Physical issues like damaged cables or faulty hardware can lead to packet loss.
3. Interference: Wireless networks are particularly susceptible to interference from other devices and obstacles.
4. Route Changes: Changes in the network routing can result in packets being lost in transition.

Effects of Packet Loss

1. Audio Issues: Missing packets in audio streams can cause glitches, dropouts, or periods of silence.
2. Video Issues: Missing packets in video streams can result in pixelation, freezing, or frame drops.
3. Overall Quality: High packet loss can degrade the overall user experience, making communication difficult and unpleasant.

Round Trip Time (RTT)

Round Trip Time (RTT) / roundTripTime is the total time for a data packet to travel from the sender to the receiver and back. It is a key metric for measuring latency in a WebRTC connection.

Higher RTT increases latency, causing noticeable delays in conversations, while lower RTT reduces latency, enabling more natural, real-time interactions.

Typical RTT values and associated experience:

Experience	RTT
Excellent	< 150ms
Acceptable	150ms - 300ms
Problematic	> 300ms

Jitter

Jitter refers to the variation in time delay between RTP packets arriving. In an ideal network, packets would arrive at evenly spaced intervals. However, in real-world networks, various factors cause packets to experience different delays, leading to jitter.

Causes of Jitter

1. Network Congestion: High traffic can cause delays as packets wait in queues before being transmitted.
2. Route Changes: Different packets may take different paths to reach the destination, resulting in variable delays.
3. Packet Loss and Retransmission: Lost packets that need to be retransmitted add to the delay variability.
4. Interference: Wireless networks, in particular, can experience interference that affects packet delivery times.

Effects of Jitter

1. Poor Audio/Video Quality: Jitter can cause audio to sound choppy or video to appear jerky.
2. Latency: Increased jitter often leads to higher overall latency, impacting real-time communication quality.
3. Buffering: To compensate for jitter, buffers are used, but excessive jitter can overwhelm buffers and cause delays.

For most real-time communication applications, jitter values can follow this :

Jitter	Range	Inference
Optimal	< 30ms	Values below 30 ms are generally considered excellent and indicate a stable network with minimal variation in packet delay.
Acceptable	30ms - 50ms	While some variation in packet arrival times is present, it is often manageable with jitter buffers and other compensatory mechanisms.
Borderline	50ms - 100ms	Values in this range might start causing noticeable issues, such as minor audio dropouts or video frame skips. Adjustments may be needed to maintain quality.
Problematic	> 100ms	Generally a sign of serious network issues, values above 100 ms can lead to significant degradation in communication quality, including choppy audio, noticeable delays, and poor video synchronization.

Total Packet Sent delay

totalPacketSendDelay metric represents the total time (in seconds) that packets have spent in the send buffer, waiting to be sent. It essentially measures the delay introduced at the sender side before the packets are transmitted over the network. This is present in outbound-rtp (kind = video/audio).

Purpose of measuring totalPacketSendDelay

1. Assessing Network Congestion:
   • High totalPacketSendDelay values can indicate network congestion or bottlenecks at the sender's end, where packets are queued up waiting to be sent.
2. Diagnosing Latency Issues:
   • Monitoring totalPacketSendDelay helps diagnose and troubleshoot latency issues. If packets are delayed before even being sent, it contributes to the overall end-to-end latency experienced by the user.
3. Optimizing Application Performance:
   • Understanding the delays in packet transmission allows developers to optimize the application. This might involve adjusting the bitrate, modifying the encoding settings, or enhancing the network infrastructure to reduce send delays.
4. Encoding and Processing Delays:
   • The time taken to encode and process media before transmission can add to the delay.
5. Hardware Performance:
   • The performance of the sender's hardware can impact how quickly packets are prepared and sent.

In an ideal scenario, totalPacketSendDelay should be as low as possible.

Available Outgoing Bitrate

availableOutgoingBitrate represents the estimated maximum bitrate that can be sent on the current network path and provides insights into the network's capacity to handle outgoing media streams. This is calculated using the underlying congestion control (gcc) algorithm. Please not that this does not indicate the exact upload bandwidth of the network connection but an approximation based on the amount of bitrate it is uploading.

Purpose of measuring availableOutgoingBitrate

1. Network Capacity Monitoring:
   • availableOutgoingBitrate helps in understanding the current network capacity for outgoing media streams. It indicates how much data can be sent per second without causing congestion or packet loss.
2. Adaptive Bitrate Management:
   • WebRTC applications often use adaptive bitrate algorithms to adjust the quality of audio and video streams based on network conditions. Monitoring availableOutgoingBitrate allows these algorithms to dynamically adjust the bitrate to match the available network capacity, ensuring smooth and high-quality media transmission.
   • Buffer: Maintaining a buffer above the required bitrate helps accommodate network fluctuations.
3. Troubleshooting and Diagnostics:
   • Sudden changes in availableOutgoingBitrate can signal network issues such as bandwidth fluctuations, congestion, or other problems. This information is valuable for diagnosing and troubleshooting network-related issues in real-time communication applications.

If this value is lesser than the upload bitrate that is configured, then there would be publish side degradations observed , i.e , reduction in bitrate and/or fps being published

Quality Limited Durations

qualityLimitationDurations is a metric that provides detailed information about the durations during which the quality of a media stream was limited due to various reasons. This metric is part of the RTCOutboundRtpStreamStats dictionary and is particularly useful for diagnosing and understanding the factors affecting the quality of a media stream. This is present in candidate-pair(state = succeeded).

Components of qualityLimitationDurations

The qualityLimitationDurations metric includes multiple sub-metrics that indicate the time spent under different types of quality limitations:

1. bandwidth: Duration during which the quality was limited due to insufficient bandwidth.
2. cpu: Duration during which the quality was limited due to high CPU usage.
3. other: Duration during which the quality was limited due to other reasons that don't fall under bandwidth or CPU constraints.

Purpose of measuring qualityLimitationDurations

1. Diagnosing Quality Issues:
   • By breaking down the durations of quality limitations into specific categories (bandwidth, CPU, other), developers can pinpoint the exact cause of quality degradation.
2. Performance Monitoring:
   • This metric helps in monitoring the performance of the WebRTC application over time, providing insights into how often and for how long the media quality is being compromised.
3. Optimization and Tuning:
   • Understanding the root causes of quality limitations allows developers to optimize and tune the WebRTC application. For example, if bandwidth is often the limiting factor, adaptive bitrate techniques can be used to manage the media quality better.

To effectively troubleshoot publisher-side issues in a WebRTC call, it’s crucial to monitor the above key metrics. One approach is to capture metrics from the getStats() API at regular intervals (e.g., every second), then aggregate or average the data over a longer period (e.g., 30 seconds) before uploading it to the server for analysis. Now that we’ve some idea about the publisher side metrics, let’s see how we can use them to debug actual webRTC calls.

Case Study

Executing a seamless video conferencing is not as straightforward as it might seem. Some issues are frequent and there is a set protocol to debugging those, but we also run into interesting problems and edge cases often. At 100ms, monitoring and addressing these metrics has become a routine part of our operations. To facilitate this process, most of the metrics mentioned earlier are directly available to customers through the 100ms Dashboard. We’ve built a powerful call quality dashboard where these statistics are plotted as time series graphs, making it easier for users to diagnose and resolve issues quickly. This data is presented at both the session and participant levels, allowing for granular insights into call performance.

We’ll take a small case study to understand how we usually debug real life call quality and experiential issues. The following snapshots are from the 100ms Peer Insights Dashboard, showing data for a participant who encountered degraded publishing quality.

Problem Statement and Impact

There was a problem where the video being seen by other participants from the participant A’s end wasn’t of good quality, which made it hard for them to view the participant A clearly.

Observations

We begin by examining the publishing bitrate graph, which indicates a significant drop in the bitrate being published around 06:18 HRS. Simultaneously, the FPS (frames per second) values also show a noticeable decline during this period.

This drop suggests that the video quality being transmitted was compromised at that time. As a result, the receivers / viewers likely experienced pixelated video due to the low bitrate and encountered choppy playback because of the reduced FPS.

This aligns with the observations shared by others participants. We’ll have to ask couple of what and why questions to understand the root cause.

What caused the sudden drop in bitrate and FPS?

To diagnose the issue further, we examine the network metrics. Key observations are as follows:

• The upload network experienced packet drops starting around 06:18 HRS, coinciding with the time of degraded video quality.
• There was a sudden spike in RTT (Round-Trip Time) values at the same timestamp, reaching as high as 6000ms.
• Packet loss occurred in bursts, with levels reaching up to 100%, as indicated in the graph.
• Although RTT values eventually returned to normal, persistent packet loss continued to affect the network.

As a result, the video publisher had to reduce both bitrate and FPS to adapt to the sudden network degradation, leading to a compromised publishing as well as viewer experience. This tells us that the culprit in question is the network, but this begs another question:

What led to the sudden network degradation, and what specific issues occurred with the network?

The network degradation appears to have been caused by congestion in the upload network. This is indicated by the jitter metric, which shows significant deviation from its normal values around the same timestamp (06:18 HRS). The increased jitter suggests that packets were arriving at the receiver end in an unevenly spaced manner, leading to packet loss.

These network issues, evidenced by the sudden spikes in jitter, RTT, and packet loss, likely caused the upload quality to deteriorate, resulting in the observed drops in bitrate and FPS. This combination of factors indicates that the peer was experiencing substantial network instability, which directly impacted the quality of the video being published.

How can we address the issue of degraded video quality caused by network instability?

While sudden and significant network drop-offs are often beyond our control, we can optimize the overall experience on consistently poor networks by adjusting parameters such as available bitrate, resolution, and FPS. These adjustments help maintain consistent, if not great video quality under challenging network conditions.

Conclusion

While we’ve focused on sender-side metrics in this blog, it’s important to remember that the receiving side plays an equally crucial role in determining the final call quality.

In our next blog, we will dive into the key metrics on the receiving / subscriber side. We will explore how these stats impact call quality and work towards developing a formula to qualitatively measure video quality at the receiver end. Understanding both sides will enable us to provide a more comprehensive approach to optimizing video call experiences.

To explore and experience these metrics firsthand, log in to the 100ms dashboard and conduct a demo call to generate real-time data. You can learn more about Peer Insights by visiting this link.