WebRTC中音视频服务质量QoS之RTT衡量网络往返时延加权平均RTT计算机制的详解
WebRTC中音视频服务质量QoS之RTT衡量网络往返时延加权平均RTT计算机制的详解
- WebRTC中音视频服务质量QoS之RTT衡量网络往返时延加权平均RTT计算机制的详解
- 前言
- 一、 RTT 网络往返时延的原理
- 1、基于发送端(SR/RR 模式)
- ①. 基本定义
- ②. 计算 RTT 网络往返时延的原理
- ③ 发送 Sender Report (SR) 协议
- SenderReport 协议的格式
- 组织SR协议
- SR和RR中都有ReportBlock数据块保存 LSR和DLSR的信息
- SR和RR中都有ReportBlock协议解析
- ④ 发送ReceiverReport(RR)协议
- ReceiverReport协议格式
- 组织 ReceiverReport(RR)数据
- 终止计算rtt往返时延 加权平均RTT计算机制
- 定时计算 WebRTC中默认1秒
- 2、基于接收端(RTCP XR 模式)
- 触发条件:接收端仅拉流(不发送媒体数据),通过 RTCP Extended Reports (XR) 扩展协议实现 RTT 探测
- 二、网络质量评估算法之时延加权平均RTT计算机制
- 三、 rtp和rtcp发送包列表数据保存时间 (WebRTC根据rtt计算的)
WebRTC专题开嗨鸭 !!!
一、 WebRTC 线程模型
1、WebRTC中线程模型和常见线程模型介绍
2、WebRTC网络PhysicalSocketServer之WSAEventselect模型使用
二、 WebRTC媒体协商
1、WebRTC媒体协商之SDP中JsepSessionDescription类结构分析
2、WebRTC媒体协商之CreatePeerConnectionFactory、CreatePeerConnection、CreateOffer
3、WebRTC之证书(certificate)生成的时机分析
4、WebRTC源码之RtpTransceiver添加视频轨道的AddTrack函数中桥接模式的流程分析
三、 WebRTC 音频数据采集
1、WebRTC源码之音频设备播放流程源码分析
2、WebRTC源码之音频设备的录制流程源码分析
四、 WebRTC 音频引擎(编解码和3A算法)
五、 WebRTC 视频数据采集
六、 WebRTC 视频引擎( 编解码)
七、 WebRTC 网络传输
1、WebRTC的ICE之STUN协议
2、WebRTC的ICE之Dtls/SSL/TLSv1.x协议详解
八、 WebRTC服务质量(Qos)
1、WebRTC中RTCP协议详解
2、WebRTC中RTP协议详解
3、WebRTC之NACK、RTX 在什么时机判断丢包发送NACK请求和RTX丢包重传
4、WebRTC源码之视频质量统计数据的数据结构分析
5、WebRTC源码之RTCPReceiver源码分析
6、WebRTC中音视频服务质量QoS之RTT衡量网络往返时延加权平均RTT计算机制的详解
九、 NetEQ
十、 Simulcast与SVC
前言
一、 RTT 网络往返时延的原理
WebRTC 提供 两种 RTT 计算模式,适应不同传输场景
1、基于发送端(SR/RR 模式)
*** 触发条件: 发送端周期性发送 Sender Report (SR),接收端回应 Receiver Report (RR) ***
①. 基本定义
DLSR 表示自接收端最后一次收到发送端 Sender Report (SR) 到生成当前 Receiver Report (RR) 的时间间隔,单位为 1/65536 秒1。若接收端未收到过 SR 报文,则 DLSR 值为零1。
②. 计算 RTT 网络往返时延的原理
在端到端通信中(以端点 A 和 B 为例):A 发送 SR:记录发送时间 t1(即 LSR,Last SR Timestamp)2。B 接收 SR:记录接收时间 last_recv_time2。B 发送 RR:计算从 last_recv_time 到当前时间的延迟(即 DLSR),并附加到 RR 报文2。A 接收 RR:根据公式 RTT = 当前时间 - LSR - DLSR 计算往返时间。
公式: R T T = T c u r r e n t − T L S R − T D L S R 65536 {RTT=T_{current} − T_ {LSR} − \frac{T_{DLSR}}{65536}} RTT=Tcurrent−TLSR−65536TDLSR (单位:秒)
参数说明:
T L S R T_ {LSR} TLSR :发送端最后一次 SR 的 NTP 时间戳(中间 32 位)3。
T D L S R T_{DLSR} TDLSR:接收端处理 SR 到生成 RR 的延迟(单位:1/65536 秒)
③ 发送 Sender Report (SR) 协议
SenderReport 协议的格式
// Sender report (SR) (RFC 3550).
// 0 1 2 3
// 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// |V=2|P| RC | PT=SR=200 | length |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 0 | SSRC of sender |
// +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
// 4 | NTP timestamp, most significant word |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 8 | NTP timestamp, least significant word |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 12 | RTP timestamp |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 16 | sender's packet count |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 20 | sender's octet count |
// 24 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
组织SR协议
std::unique_ptr<rtcp::RtcpPacket> RTCPSender::BuildSR(const RtcpContext& ctx) {// Timestamp shouldn't be estimated before first media frame.RTC_DCHECK_GE(last_frame_capture_time_ms_, 0);// The timestamp of this RTCP packet should be estimated as the timestamp of// the frame being captured at this moment. We are calculating that// timestamp as the last frame's timestamp + the time since the last frame// was captured.int rtp_rate = rtp_clock_rates_khz_[last_payload_type_];if (rtp_rate <= 0) {rtp_rate =(audio_ ? kBogusRtpRateForAudioRtcp : kVideoPayloadTypeFrequency) /1000;}// Round now_us_ to the closest millisecond, because Ntp time is rounded// when converted to milliseconds,uint32_t rtp_timestamp =timestamp_offset_ + last_rtp_timestamp_ +((ctx.now_us_ + 500) / 1000 - last_frame_capture_time_ms_) * rtp_rate;rtcp::SenderReport* report = new rtcp::SenderReport();report->SetSenderSsrc(ssrc_);report->SetNtp(TimeMicrosToNtp(ctx.now_us_));report->SetRtpTimestamp(rtp_timestamp);report->SetPacketCount(ctx.feedback_state_.packets_sent);report->SetOctetCount(ctx.feedback_state_.media_bytes_sent);// TODO@chensong 2025-03-15 获取当前发送 report->SetReportBlocks(CreateReportBlocks(ctx.feedback_state_));return std::unique_ptr<rtcp::RtcpPacket>(report);
}
SR和RR中都有ReportBlock数据块保存 LSR和DLSR的信息
// From RFC 3550, RTP: A Transport Protocol for Real-Time Applications.
//
// RTCP report block (RFC 3550).
//
// 0 1 2 3
// 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
// +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
// 0 | SSRC_1 (SSRC of first source) |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 4 | fraction lost | cumulative number of packets lost |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 8 | extended highest sequence number received |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 12 | interarrival jitter |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 16 | last SR (LSR) |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// 20 | delay since last SR (DLSR) |
// 24 +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
SR和RR中都有ReportBlock协议解析
last_sr_ :发送端发送时间
delay_since_last_sr_ : 是远端最后接受SR或者RR包的时间
bool ReportBlock::Parse(const uint8_t* buffer, size_t length)
{RTC_DCHECK(buffer != nullptr);if (length < ReportBlock::kLength){RTC_LOG(LS_ERROR) << "Report Block should be 24 bytes long";return false;}// 接收到的媒体源ssrcsource_ssrc_ = ByteReader<uint32_t>::ReadBigEndian(&buffer[0]);// TODO@chensong 2022-10-19 丢包率 fraction_lost/**TODO@chensong 2023-03-07 某时刻收到的有序包的数量Count = transmitted-retransmitte,当前时刻为Count2,上一时刻为Count1;接收端以一定的频率发送RTCP包(RR、REMB、NACK等)时,会统计两次发送间隔之间(fraction)的接收包信息。接收端发送的RR包中包含两个丢包:一个是fraction_lost,是两次统计间隔间的丢包率(以256为基数换算成8bit)。一个是cumulative number of packets lost,是总的累积丢包。 **/fraction_lost_ = buffer[4];// 接收开始丢包总数, 迟到包不算丢包,重传有可以导致负数cumulative_lost_ = ByteReader<int32_t, 3>::ReadBigEndian(&buffer[5]);// 低16位表示收到的最大seq,高16位表示seq循环次数extended_high_seq_num_ = ByteReader<uint32_t>::ReadBigEndian(&buffer[8]);// rtp包到达时间间隔的统计方差jitter_ = ByteReader<uint32_t>::ReadBigEndian(&buffer[12]);// ntp时间戳的中间32位last_sr_ = ByteReader<uint32_t>::ReadBigEndian(&buffer[16]);// 记录上一个接收SR的时间与上一个发送SR的时间差delay_since_last_sr_ = ByteReader<uint32_t>::ReadBigEndian(&buffer[20]);return true;
}
④ 发送ReceiverReport(RR)协议
ReceiverReport协议格式
// RTCP receiver report (RFC 3550).
//
// 0 1 2 3
// 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// |V=2|P| RC | PT=RR=201 | length |
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
// | SSRC of packet sender |
// +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
// | report block(s) |
// | .... |
组织 ReceiverReport(RR)数据
在RTCPSender类中BuildRR方法中调用 GetFeedbackState方法获取 ReportBlock数据
调用流程
RTCPSender类BuildRR —> ModuleRtpRtcpImpl::GetFeedbackState获取 remote_sender_rtp_time_(远端发送时间)和 last_received_sr_ntp_ (最后一次接受时间)
—>LastReceivedNTP 方法调用NTP方法
–>RTCPReceiver类NTP 获取 remote_sender_rtp_time_(远端发送时间)和 last_received_sr_ntp_ (最后一次接受时间)
std::unique_ptr<rtcp::RtcpPacket> RTCPSender::BuildRR(const RtcpContext& ctx) {rtcp::ReceiverReport* report = new rtcp::ReceiverReport();report->SetSenderSsrc(ssrc_);// TODO@chensong 2025-03-15 rtp_rtcp_impl.cc -> ModuleRtpRtcpImpl::GetFeedbackStatereport->SetReportBlocks(CreateReportBlocks(ctx.feedback_state_));return std::unique_ptr<rtcp::RtcpPacket>(report);
}// TODO(pbos): Handle media and RTX streams separately (separate RTCP
// feedbacks).
RTCPSender::FeedbackState ModuleRtpRtcpImpl::GetFeedbackState() {RTCPSender::FeedbackState state;// This is called also when receiver_only is true. Hence below// checks that rtp_sender_ exists.if (rtp_sender_) {StreamDataCounters rtp_stats;StreamDataCounters rtx_stats;rtp_sender_->GetDataCounters(&rtp_stats, &rtx_stats);state.packets_sent =rtp_stats.transmitted.packets + rtx_stats.transmitted.packets;state.media_bytes_sent = rtp_stats.transmitted.payload_bytes +rtx_stats.transmitted.payload_bytes;state.send_bitrate = rtp_sender_->BitrateSent();}state.module = this;// TODO@chensong 2025-03-15 获取远端发送信息包时间 和当前最后接收一包记录时间LastReceivedNTP(&state.last_rr_ntp_secs, &state.last_rr_ntp_frac,&state.remote_sr);state.last_xr_rtis = rtcp_receiver_.ConsumeReceivedXrReferenceTimeInfo();return state;
}bool RTCPReceiver::NTP(uint32_t* received_ntp_secs,uint32_t* received_ntp_frac,uint32_t* rtcp_arrival_time_secs,uint32_t* rtcp_arrival_time_frac,uint32_t* rtcp_timestamp) const {rtc::CritScope lock(&rtcp_receiver_lock_);if (!last_received_sr_ntp_.Valid()) {return false;}// TODO@chensong 2025-03-15 last_rr_ntp_frac 发送时间戳// NTP from incoming SenderReport.if (received_ntp_secs) {*received_ntp_secs = remote_sender_ntp_time_.seconds();}if (received_ntp_frac) {*received_ntp_frac = remote_sender_ntp_time_.fractions();}// Rtp time from incoming SenderReport.// TODO@chensong 2025-03-15 远端接受最后一个rtp包的时间if (rtcp_timestamp) {*rtcp_timestamp = remote_sender_rtp_time_;}// Local NTP time when we received a RTCP packet with a send block.// TODO@chensong 2025-03-15 本地接受最后一个rtcp包的时间if (rtcp_arrival_time_secs) {*rtcp_arrival_time_secs = last_received_sr_ntp_.seconds();}if (rtcp_arrival_time_frac) {*rtcp_arrival_time_frac = last_received_sr_ntp_.fractions();}return true;
}
// 接收SenderReport包信息
void RTCPReceiver::HandleSenderReport(const CommonHeader& rtcp_block,PacketInformation* packet_information) {rtcp::SenderReport sender_report;if (!sender_report.Parse(rtcp_block)) {++num_skipped_packets_;return;}const uint32_t remote_ssrc = sender_report.sender_ssrc();packet_information->remote_ssrc = remote_ssrc;UpdateTmmbrRemoteIsAlive(remote_ssrc);// Have I received RTP packets from this party?if (remote_ssrc_ == remote_ssrc) {// Only signal that we have received a SR when we accept one.packet_information->packet_type_flags |= kRtcpSr;// TODO@chensong 2025-03-15 SR => RR remote_sender_ntp_time_ = sender_report.ntp();remote_sender_rtp_time_ = sender_report.rtp_timestamp();last_received_sr_ntp_ = TimeMicrosToNtp(clock_->TimeInMicroseconds());} else {// We will only store the send report from one source, but// we will store all the receive blocks.packet_information->packet_type_flags |= kRtcpRr;}for (const rtcp::ReportBlock& report_block : sender_report.report_blocks()) {HandleReportBlock(report_block, packet_information, remote_ssrc);}
}
终止计算rtt往返时延 加权平均RTT计算机制
定时计算 WebRTC中默认1秒
在ModuleRtpRtcpImpl类中Process方法中统计 加权平均RTT计算机制
// Process any pending tasks such as timeouts (non time critical events).
void ModuleRtpRtcpImpl::Process() {const int64_t now = clock_->TimeInMilliseconds();next_process_time_ = now + kRtpRtcpMaxIdleTimeProcessMs;if (rtp_sender_) {if (now >= last_bitrate_process_time_ + kRtpRtcpBitrateProcessTimeMs) {rtp_sender_->ProcessBitrate();last_bitrate_process_time_ = now;next_process_time_ =std::min(next_process_time_, now + kRtpRtcpBitrateProcessTimeMs);}}bool process_rtt = now >= last_rtt_process_time_ + kRtpRtcpRttProcessTimeMs;if (rtcp_sender_.Sending()) {// Process RTT if we have received a report block and we haven't// processed RTT for at least |kRtpRtcpRttProcessTimeMs| milliseconds.if (rtcp_receiver_.LastReceivedReportBlockMs() > last_rtt_process_time_ &&process_rtt) {std::vector<RTCPReportBlock> receive_blocks;rtcp_receiver_.StatisticsReceived(&receive_blocks);int64_t max_rtt = 0;for (std::vector<RTCPReportBlock>::iterator it = receive_blocks.begin();it != receive_blocks.end(); ++it) {int64_t rtt = 0;rtcp_receiver_.RTT(it->sender_ssrc, &rtt, NULL, NULL, NULL);max_rtt = (rtt > max_rtt) ? rtt : max_rtt;}// Report the rtt.if (rtt_stats_ && max_rtt != 0)rtt_stats_->OnRttUpdate(max_rtt);}// Verify receiver reports are delivered and the reported sequence number// is increasing.if (rtcp_receiver_.RtcpRrTimeout()) {RTC_LOG_F(LS_WARNING) << "Timeout: No RTCP RR received.";} else if (rtcp_receiver_.RtcpRrSequenceNumberTimeout()) {RTC_LOG_F(LS_WARNING) << "Timeout: No increase in RTCP RR extended ""highest sequence number.";}if (remote_bitrate_ && rtcp_sender_.TMMBR()) {unsigned int target_bitrate = 0;std::vector<unsigned int> ssrcs;if (remote_bitrate_->LatestEstimate(&ssrcs, &target_bitrate)) {if (!ssrcs.empty()) {target_bitrate = target_bitrate / ssrcs.size();}rtcp_sender_.SetTargetBitrate(target_bitrate);}}} else {// Report rtt from receiver.if (process_rtt) {int64_t rtt_ms;if (rtt_stats_ && rtcp_receiver_.GetAndResetXrRrRtt(&rtt_ms)) {rtt_stats_->OnRttUpdate(rtt_ms);}}}// Get processed rtt.if (process_rtt) {last_rtt_process_time_ = now;next_process_time_ = std::min(next_process_time_, last_rtt_process_time_ + kRtpRtcpRttProcessTimeMs);if (rtt_stats_) {// TODO@chensong 2025-03-15 1秒更新一次 rtt 公式/*TODO@chensong 2025-03-15 加权平均RTT计算机制在实时通信场景(如WebRTC)中,RTT(往返时延)的平滑计算对网络状态感知和拥塞控制至关重要。通过 加权移动平均(Weighted Moving Average) 对RTT值进行动态调整,可有效平衡历史数据与实时测量值的影响,抑制短期波动带来的干扰。以下是核心实现逻辑:1. 公式定义计算方式:新平均RTT由 历史平均值(old_avg) 与 最新测量值(new_sample) 按权重合成,公式为:textCopy Codeavg_rtt = 0.7 * old_avg + 0.3 * new_sample 其中,历史数据权重为70%(0.7),新样本权重为30%(0.3)23。数学意义:旧值主导(70%):确保长期趋势稳定,避免偶发延迟突变(如网络抖动)对整体估计的过度影响23。新值补充(30%):快速响应网络状态的渐进变化(如带宽增减或路由切换)*/// Make sure we have a valid RTT before setting.int64_t last_rtt = rtt_stats_->LastProcessedRtt();if (last_rtt >= 0)set_rtt_ms(last_rtt);}}if (rtcp_sender_.TimeToSendRTCPReport())rtcp_sender_.SendRTCP(GetFeedbackState(), kRtcpReport);if (TMMBR() && rtcp_receiver_.UpdateTmmbrTimers()) {rtcp_receiver_.NotifyTmmbrUpdated();}
}
2、基于接收端(RTCP XR 模式)
触发条件:接收端仅拉流(不发送媒体数据),通过 RTCP Extended Reports (XR) 扩展协议实现 RTT 探测
实现步骤:
-
网关发送 RRTR 报文(含 NTP 时间戳 T R R T R T_{RRTR} TRRTR)
-
接收端回复 DLRR 报文,包含
- 原 T R R T R T_{RRTR} TRRTR (即为LRR)
- 处理延迟 T D L S R T_{DLSR} TDLSR(接收 RRTR 到发送 DLRR 的时间)
-
网关计算公式
R T T = T c u r r e n t RTT = {T_{current}} RTT=Tcurrent - T T R R {T_{TRR}} TTRR - T D L S R {T_{DLSR}} TDLSR
二、网络质量评估算法之时延加权平均RTT计算机制
加权平均RTT计算机制
在实时通信场景(如WebRTC)中,RTT(往返时延)的平滑计算对网络状态感知和拥塞控制至关重要。通过 加权移动平均(Weighted Moving Average)
对RTT值进行动态调整,可有效平衡历史数据与实时测量值的影响,抑制短期波动带来的干扰。以下是核心实现逻辑:
1. 公式定义计算方式:新平均RTT由 历史平均值(old_avg) 与 最新测量值(new_sample) 按权重合成,公式为:avg_rtt = 0.7 * old_avg + 0.3 * new_sample 其中,历史数据权重为70%(0.7),新样本权重为30%(0.3)23。数学意义:旧值主导(70%):确保长期趋势稳定,避免偶发延迟突变(如网络抖动)对整体估计的过度影响23。新值补充(30%):快速响应网络状态的渐进变化(如带宽增减或路由切换)
三、 rtp和rtcp发送包列表数据保存时间 (WebRTC根据rtt计算的)
void RtpPacketHistory::CullOldPackets(int64_t now_ms)
{//TODO@chensong 2025-03-15 比如NACK(否定确认)或ARQ(自动重传请求)中的缓冲区管理策略有关。// 根据 rtt 放弃 rtp包 // 公式 : 淘汰时间 = 3 × max(基准时间, 3 × 当前RTT)// 基准时间通常为 1000ms(兜底值,防止 RTT 过小导致缓存不足)int64_t packet_duration_ms = std::max(kMinPacketDurationRtt * rtt_ms_, kMinPacketDurationMs);while (!packet_history_.empty()){auto stored_packet_it = packet_history_.find(*start_seqno_);RTC_DCHECK(stored_packet_it != packet_history_.end());if (packet_history_.size() >= kMaxCapacity /* 9600*/) {// We have reached the absolute max capacity, remove one packet// unconditionally.RemovePacket(stored_packet_it);continue;}const StoredPacket& stored_packet = stored_packet_it->second;if (!stored_packet.send_time_ms) {// Don't remove packets that have not been sent.return;}if (*stored_packet.send_time_ms + packet_duration_ms > now_ms) {// Don't cull packets too early to avoid failed retransmission requests.return;}if (packet_history_.size() >= number_to_store_ ||(mode_ == StorageMode::kStoreAndCull && *stored_packet.send_time_ms + (packet_duration_ms * kPacketCullingDelayFactor) <= now_ms)) {// Too many packets in history, or this packet has timed out. Remove it// and continue.RemovePacket(stored_packet_it);}else {// No more packets can be removed right now.return;}}
}