Android平台GB28181设备接入模块开发填坑指南

技术背景

为什么要开发Android平台GB28181设备接入模块?这个问题不再赘述,在做Android平台GB28181客户端的时候,媒体数据这块,我们已经有了很好的积累,因为在此之前,我们就开发了非常成熟的RTMP推送、轻量级RTSP服务、录像模块、针对音视频的对接处理单元。这让我们在做Android平台GB28181设备接入模块的时候,可以有更多的精力在信令交互和国标平台对接。

好多开发者会觉得,GB28181设备接入模块有啥好做的?不就找个开源的SIP信令,视频编码ps打包下投递到国标平台就好了吗?

事实上,当回头看看开发的功能时,就会觉得,一两个月的东西,仅就可以作为项目交付或demo使用,并不会有多大的商业价值,因为需要解决的问题实在太多了。

  •  [视频格式]H.264/H.265(Android H.265硬编码);
  •  [音频格式]G.711 A律、AAC;
  •  [音量调节]Android平台采集端支持实时音量调节;
  •  [H.264硬编码]支持H.264特定机型硬编码;
  •  [H.265硬编码]支持H.265特定机型硬编码;
  •  [软硬编码参数配置]支持gop间隔、帧率、bit-rate设置;
  •  [软编码参数配置]支持软编码profile、软编码速度、可变码率设置;
  • 支持纯视频、音视频PS打包传输;
  • 支持RTP OVER UDP和RTP OVER TCP被动模式(TCP媒体流传输客户端);
  • 支持信令通道网络传输协议TCP/UDP设置;
  • 支持注册、注销,支持注册刷新及注册有效期设置;
  • 支持设备目录查询应答;
  • 支持心跳机制,支持心跳间隔、心跳检测次数设置;
  • 支持移动设备位置(MobilePosition)订阅和通知;
  • 支持语音广播;
  • 支持语音对讲;
  • 支持历史视音频文件检索;
  • 支持历史视音频文件下载;
  • 支持历史视音频文件回放;
  • 支持云台控制和预置位查询;
  •  [实时水印]支持动态文字水印、png水印;
  •  [镜像]Android平台支持前置摄像头实时镜像功能;
  •  [实时静音]支持实时静音/取消静音;
  •  [实时快照]支持实时快照;
  •  [降噪]支持环境音、手机干扰等引起的噪音降噪处理、自动增益、VAD检测;
  •  [外部编码前视频数据对接]支持YUV数据对接;
  •  [外部编码前音频数据对接]支持PCM对接;
  •  [外部编码后视频数据对接]支持外部H.264数据对接;
  •  [外部编码后音频数据对接]外部AAC数据对接;
  •  [扩展录像功能]支持和录像模块组合使用,录像相关功能。

技术实现

[视频格式]H.264/H.265(Android H.265硬编码)

目前GB28181-2022已经明确表示支持H.265,GB28181设备接入这块,如果需要有好的画质,编码算法这块,一定需要做好,Android端除了低分辨率软编外,超过1280*720,一般建议硬编码。

	 /*** Set Video H.264 HW Encoder, if support HW encoder, it will return 0(设置H.264硬编码)* * @param kbps: the kbps of different resolution.* * @return {0} if successful*/public native int SetSmartPublisherVideoHWEncoder(long handle, int kbps);/*** Set Video H.265(hevc) hardware encoder, if support H.265(hevc) hardware encoder, it will return 0(设置H.265硬编码)** @param kbps: the kbps of different resolution.** @return {0} if successful*/public native int SetSmartPublisherVideoHevcHWEncoder(long handle, int kbps);
硬编码参数设置
	/** 设置视频硬编码码率控制模式* @param hw_bitrate_mode: -1表示使用默认值, 不设置也会使用默认值, 0:CQ, 1:VBR, 2:CBR, 3:CBR_FD, 请参考:android.media.MediaCodecInfo.EncoderCapabilities* 注意硬编码和手机硬件有关,多数手机只支持部分码率模式, 另外硬编码设备差异很大,不同设备同一码率控制模式效果可能不一样* @return {0} if successful*/public native int SetVideoHWEncoderBitrateMode(long handle, int hw_bitrate_mode);/** 设置视频硬编码复杂度, 安卓5.0及以上支持* @param hw_complexity: -1表示不设置, 请参考:android.media.MediaCodecInfo.EncoderCapabilities.getComplexityRange() 和 android.media.MediaFormat.KEY_COMPLEXITY* 注意硬编码和手机硬件有关,部分手机可能不支持此设置* @return {0} if successful*/public native int SetVideoHWEncoderComplexity(long handle, int hw_complexity);/** 设置视频硬编码质量, 安卓9及以上支持, 仅当硬编码器码率控制模式(BitrateMode)是CQ(constant-quality mode)时才有效* @param hw_quality: -1表示不设置, 请参考:android.media.MediaCodecInfo.EncoderCapabilities.getQualityRange() 和 android.media.MediaFormat.KEY_QUALITY* 注意硬编码和手机硬件有关,部分手机可能不支持此设置* @return {0} if successful*/public native int SetVideoHWEncoderQuality(long handle, int hw_quality);/** 设置H.264硬编码Profile, 安卓7及以上支持* @param hw_avc_profile: 0表示使用默认值, 0x01: Baseline, 0x02: Main, 0x08: High, 0x10000: ConstrainedBaseline, 0x80000: ConstrainedHigh;* 注意: ConstrainedBaseline 和 ConstrainedHigh 可能多数设备不支持,* H.264推荐使用 High 或者 ConstrainedHigh, 如果您使用的手机硬解码解不了,那还是设置Baseline* 如果设置的Profile硬编码器不支持,应编码器会使用默认值* 具体参考:android.media.MediaCodecInfo.CodecProfileLevel* @return {0} if successful*/public native int SetAVCHWEncoderProfile(long handle, int hw_avc_profile);/** 设置H.264硬编码Level, 这个只有在设置了Profile的情况下才有效, 安卓7及以上支持* @param hw_avc_level: 0表示使用默认值, 0x100: Level3, 0x200: Level3.1, 0x400: Level3.2,* 0x800: Level4, 0x1000: Level4.1, 0x2000: Level4.2,* 0x4000: Level5, 0x8000: Level5.1,  0x10000: Level5.2,* 0x20000: Level6, 0x40000: Level6.1,  0x80000: Level6.2,* 如果设置的level太高硬编码器不支持,SDK内部会做相应调整* 注意: 640*480@25fps最小支持的是Level3, 720p最小支持的是Level3.1, 1080p最小支持的是Level4* 具体参考:android.media.MediaCodecInfo.CodecProfileLevel* @return {0} if successful*/public native int SetAVCHWEncoderLevel(long handle, int hw_avc_level);/** 设置视频硬编码最大码率, 安卓没有相关文档说明, 所以不建议设置,* @param hw_max_bitrate: 每秒最大码率, 单位bps* @return {0} if successful*/public native int SetVideoHWEncoderMaxBitrate(long handle, long hw_max_bitrate);
[音频格式]G.711 A律、AAC
    /*** Set audio encoder type(设置音频编码类型)* * @param type: if with 1:AAC, if with 2: SPEEX, if with 3: PCMA* * @return {0} if successful*/public native int SmartPublisherSetAudioCodecType(long handle, int type);/*** Set audio encoder bit-rate(设置音频编码码率), 当前只对AAC编码有效** @param kbit_rate: 码率(单位是kbps), 如果是0的话将使用默认码率, 必须大于等于0** @return {0} if successful*/public native int SmartPublisherSetAudioBitRate(long handle, int kbit_rate);
[音量调节]Android平台采集端支持实时音量调节
	/*** 设置输入音量, 这个接口一般不建议调用, 在一些特殊情况下可能会用, 一般不建议放大音量** @param index: 一般是0和1, 如果没有混音的只用0, 有混音的话, 0,1分别设置音量** @param volume: 音量,默认是1.0,范围是[0.0, 5.0], 设置成0静音, 1音量不变** @return {0} if successful*/public native int SmartPublisherSetInputAudioVolume(long handle, int index, float volume);
[软硬编码参数配置]支持gop间隔、帧率、bit-rate设置,支持软编码profile、软编码速度、可变码率设置
	/*** Set software encode vbr mode(软编码可变码率).** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** is_enable_vbr: if 0: NOT enable vbr mode, 1: enable vbr** video_quality: vbr video quality, range with (1,50), default 23** vbr_max_kbitrate: vbr max encode bit-rate(kbps)** @return {0} if successful*/public native int SmartPublisherSetSwVBRMode(long handle, int is_enable_vbr, int video_quality, int vbr_max_kbitrate);/*** Set gop interval(设置I帧间隔)** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** gopInterval: encode I frame interval, the value always > 0** @return {0} if successful*/public native int SmartPublisherSetGopInterval(long handle, int gopInterval);/*** Set software encode video bit-rate(设置视频软编码bit-rate)** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** avgBitRate: average encode bit-rate(kbps)* * maxBitRate: max encode bit-rate(kbps)** @return {0} if successful*/public native int SmartPublisherSetSWVideoBitRate(long handle, int avgBitRate, int maxBitRate);/*** Set fps(设置帧率)** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** fps: the fps of video, range with (1,25).** @return {0} if successful*/public native int SmartPublisherSetFPS(long handle, int fps);/*** Set software video encoder profile(设置视频编码profile).** <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>** profile: the software video encoder profile, range with (1,3).* * 1: baseline profile* 2: main profile* 3: high profile** @return {0} if successful*/public native int SmartPublisherSetSWVideoEncoderProfile(long handle, int profile);/*** Set software video encoder speed(设置视频软编码编码速度)* * <pre>please set before SmartPublisherStart while after SmartPublisherOpen.</pre>* * @param speed: range with(1, 6), the default speed is 6. * * if with 1, CPU is lowest.* if with 6, CPU is highest.* * @return {0} if successful*/public native int SmartPublisherSetSWVideoEncoderSpeed(long handle, int speed);
信令通道网络传输协议TCP/UDP设置
gb28181_agent_.setTransportProtocol(gb28181_sip_trans_protocol_==0?"UDP":"TCP");
支持注册、注销,支持注册刷新及注册有效期设置
private int gb28181_reg_expired_           = 3600; // 注册有效期时间最小3600秒// GB28181配置
gb28181_agent_.config(gb28181_reg_expired_, gb28181_heartbeat_interval_, gb28181_heartbeat_count_);@Override
public void ntsRegisterOK(String dateString) {Log.i(TAG, "ntsRegisterOK Date: " + (dateString!= null? dateString : ""));
}@Override
public void ntsRegisterTimeout() {Log.e(TAG, "ntsRegisterTimeout");
}@Override
public void ntsRegisterTransportError(String errorInfo) {Log.e(TAG, "ntsRegisterTransportError error:" + (errorInfo != null?errorInfo :""));
}
支持心跳机制,支持心跳间隔、心跳检测次数设置
private int gb28181_heartbeat_interval_    = 20; // 心跳间隔GB28181默认是60, 目前调整到20秒
private int gb28181_heartbeat_count_       = 3; // 心跳间隔3次失败,表示和服务器断开了@Override
public void ntsOnHeartBeatException(int exceptionCount,  String lastExceptionInfo) {Log.e(TAG, "ntsOnHeartBeatException heart beat timeout count reached, count:" + exceptionCount+", exception info:" + (lastExceptionInfo!=null?lastExceptionInfo:""));// 停止信令, 然后重启handler_.postDelayed(new Runnable() {@Overridepublic void run() {Log.i(TAG, "gb28281_heart_beart_timeout");record_executor_.cancel_tasks();stopRecordDownloads(true);stopPlaybacks(true);stopAudioPlayer();destoryRTPReceiver();gb_broadcast_source_id_ = null;gb_broadcast_target_id_ = null;btnGB28181AudioBroadcast.setText("GB28181语音广播");btnGB28181AudioBroadcast.setEnabled(false);stopGB28181Stream();destoryRTPSender();if (gb28181_agent_ != null) {gb28181_agent_.terminateAllAudioBroadcasts(true);gb28181_agent_.terminateAllPlays(true);Log.i(TAG, "gb28281_heart_beart_timeout sip stop");gb28181_agent_.stop();String local_ip_addr = IPAddrUtils.getIpAddress(context_);if (local_ip_addr != null && !local_ip_addr.isEmpty() ) {Log.i(TAG, "gb28281_heart_beart_timeout get local ip addr: " + local_ip_addr);gb28181_agent_.setLocalAddress(local_ip_addr);}record_executor_.cancel_tasks();initPlaybacks(null);initRecordDownloads(null);Log.i(TAG, "gb28281_heart_beart_timeout sip start");gb28181_agent_.start();}}},0);
}
支持移动设备位置(MobilePosition)订阅和通知
com.gb.ntsignalling.Device gb_device = new com.gb.ntsignalling.Device("34020000001380000001", "安卓测试设备", Build.MANUFACTURER, Build.MODEL,"宇宙","火星1","火星", true);if (mLongitude != null && mLatitude != null) {com.gb.ntsignalling.DevicePosition device_pos = new com.gb.ntsignalling.DevicePosition();device_pos.setTime(mLocationTime);device_pos.setLongitude(mLongitude);device_pos.setLatitude(mLatitude);gb_device.setPosition(device_pos);gb_device.setSupportMobilePosition(true); // 设置支持移动位置上报
}@Override
public void ntsOnDevicePositionRequest(String deviceId, int interval) {handler_.postDelayed(new Runnable() {@Overridepublic void run() {getLocation(context_);Log.v(TAG, "ntsOnDevicePositionRequest, deviceId:" + this.device_id_ + ", Longitude:" + mLongitude+ ", Latitude:" + mLatitude + ", Time:" + mLocationTime);if (mLongitude != null && mLatitude != null) {com.gb.ntsignalling.DevicePosition device_pos = new com.gb.ntsignalling.DevicePosition();device_pos.setTime(mLocationTime);device_pos.setLongitude(mLongitude);device_pos.setLatitude(mLatitude);if (gb28181_agent_ != null ) {gb28181_agent_.updateDevicePosition(device_id_, device_pos);}}}private String device_id_;private int interval_;public Runnable set(String device_id, int interval) {this.device_id_ = device_id;this.interval_ = interval;return this;}}.set(deviceId, interval),0);
}
支持语音广播和语音对讲
@Override
public void ntsOnAudioBroadcast(String commandFromUserName, String commandFromUserNameAtDomain, String sourceID, String targetID) {handler_.postDelayed(new Runnable() {@Overridepublic void run() {Log.i(TAG, "ntsOnAudioBroadcastPlay, fromFromUserName:" + command_from_user_name_+ " FromUserNameAtDomain:" + command_from_user_name_at_domain_+ " sourceID:" + source_id_ + ", targetID:" + target_id_);stopAudioPlayer();destoryRTPReceiver();if (gb28181_agent_ != null ) {String local_ip_addr = IPAddrUtils.getIpAddress(context_);boolean is_tcp = true; // 考虑到跨网段, 默认用TCP传输rtp包rtp_receiver_handle_ = lib_player_.CreateRTPReceiver(0);if (rtp_receiver_handle_ != 0 ) {lib_player_.SetRTPReceiverTransportProtocol(rtp_receiver_handle_, is_tcp?1:0);lib_player_.SetRTPReceiverIPAddressType(rtp_receiver_handle_, 0);if (0 == lib_player_.CreateRTPReceiverSession(rtp_receiver_handle_, 0) ) {int local_port = lib_player_.GetRTPReceiverLocalPort(rtp_receiver_handle_);boolean ret = gb28181_agent_.inviteAudioBroadcast(command_from_user_name_,command_from_user_name_at_domain_,source_id_, target_id_, "IP4", local_ip_addr, local_port, is_tcp?"TCP/RTP/AVP":"RTP/AVP");if (!ret ) {destoryRTPReceiver();btnGB28181AudioBroadcast.setText("GB28181语音广播");}else {btnGB28181AudioBroadcast.setText("GB28181语音广播呼叫中");}} else {destoryRTPReceiver();btnGB28181AudioBroadcast.setText("GB28181语音广播");}}}}private String command_from_user_name_;private String command_from_user_name_at_domain_;private String source_id_;private String target_id_;public Runnable set(String command_from_user_name, String command_from_user_name_at_domain, String source_id, String target_id) {this.command_from_user_name_ = command_from_user_name;this.command_from_user_name_at_domain_ = command_from_user_name_at_domain;this.source_id_ = source_id;this.target_id_ = target_id;return this;}}.set(commandFromUserName, commandFromUserNameAtDomain, sourceID, targetID),0);
}@Override
public void ntsOnInviteAudioBroadcastException(String sourceID, String targetID, String errorInfo) {handler_.postDelayed(new Runnable() {@Overridepublic void run() {Log.i(TAG, "ntsOnInviteAudioBroadcastException, sourceID:" + source_id_ + ", targetID:" + target_id_);destoryRTPReceiver();btnGB28181AudioBroadcast.setText("GB28181语音广播");}private String source_id_;private String target_id_;public Runnable set(String source_id, String target_id) {this.source_id_ = source_id;this.target_id_ = target_id;return this;}}.set(sourceID, targetID),0);
}@Override
public void ntsOnInviteAudioBroadcastTimeout(String sourceID, String targetID) {handler_.postDelayed(new Runnable() {@Overridepublic void run() {Log.i(TAG, "ntsOnInviteAudioBroadcastTimeout, sourceID:" + source_id_ + ", targetID:" + target_id_);destoryRTPReceiver();btnGB28181AudioBroadcast.setText("GB28181语音广播");}private String source_id_;private String target_id_;public Runnable set(String source_id, String target_id) {this.source_id_ = source_id;this.target_id_ = target_id;return this;}}.set(sourceID, targetID),0);
}
支持历史视音频文件检索、支持历史视音频文件下载和回放
/*** Author: daniusdk.com*/
package com.gb.ntsignalling;public interface GBSIPAgent {void addDownloadListener(GBSIPAgentDownloadListener downloadListener);void removeDownloadListener(GBSIPAgentDownloadListener removeListener);/**响应Invite Download 200 OK*/boolean respondDownloadInviteOK(long id, String deviceId, String startTime, String stopTime, MediaSessionDescription localMediaDescription);/**响应Invite Download 其他状态码*/boolean respondDownloadInvite(int statusCode, long id, String deviceId, String startTime, String stopTime);/** 媒体流发送者在文件下载结束后发Message消息通知SIP服务器回文件已发送完成* notifyType 必须是"121“*/boolean notifyDownloadMediaStatus(long id, String deviceId, String startTime, String stopTime, String notifyType);/**终止Download会话*/void terminateDownload(long id, String deviceId, String startTime, String stopTime, boolean isSendBYE);/**终止所有Download会话*/void terminateAllDownloads(boolean isSendBYE);}package com.gb.ntsignalling;public interface GBSIPAgent {void addPlaybackListener(GBSIPAgentPlaybackListener playbackListener);void removePlaybackListener(GBSIPAgentPlaybackListener playbackListener);/**响应Invite Playback 200 OK*/boolean respondPlaybackInviteOK(long id, String deviceId, String startTime, String stopTime, MediaSessionDescription localMediaDescription);/**响应Invite Playback 其他状态码*/boolean respondPlaybackInvite(int statusCode, long id, String deviceId);/** 媒体流发送者在回放结束后发Message消息通知SIP服务器回放文件已发送完成* notifyType 必须是"121"*/boolean notifyPlaybackMediaStatus(long id, String deviceId, String notifyType);/**终止Playback会话*/void terminatePlayback(long id, String deviceId, boolean isSendBYE);/**终止所有Playback会话*/void terminateAllPlaybacks(boolean isSendBYE);
}/**
* 信令Playback Listener
*/
package com.gb.ntsignalling;public interface GBSIPAgentPlaybackListener {/**收到s=Playback的历史回放Invite*/void ntsOnInvitePlayback(long id, String deviceId, SessionDescription sessionDescription);/**发送Playback invite response 异常*/void ntsOnPlaybackInviteResponseException(long id, String deviceId, int statusCode, String errorInfo);/** 收到CANCEL Playback INVITE请求*/void ntsOnCancelPlayback(long id, String deviceId);/** 收到Ack*/void ntsOnAckPlayback(long id, String deviceId);/** 播放命令*/void ntsOnPlaybackMANSRTSPPlayCommand(long id, String deviceId);/** 暂停命令*/void ntsOnPlaybackMANSRTSPPauseCommand(long id, String deviceId);/** 快进/慢进命令*/void ntsOnPlaybackMANSRTSPScaleCommand(long id, String deviceId, double scale);/** 随机拖动命令*/void ntsOnPlaybackMANSRTSPSeekCommand(long id, String deviceId, double position_sec);/** 停止命令*/void ntsOnPlaybackMANSRTSPTeardownCommand(long id, String deviceId);/** 收到Bye*/void ntsOnByePlayback(long id, String deviceId);/** 不是在收到BYE Message情况下, 终止Playback*/void ntsOnTerminatePlayback(long id, String deviceId);/** Playback会话对应的对话终止, 一般不会触发这个回调,目前只有在响应了200K, 但在64*T1时间后还没收到ACK,才可能会出发收到这个, 请做相关清理处理*/void ntsOnPlaybackDialogTerminated(long id, String deviceId);
}/**
* 部分JNI接口, rtp ps 打包发送等代码C++实现
*/public class SmartPublisherJniV2 {/*** Open publisher(启动推送实例)** @param ctx: get by this.getApplicationContext()* * @param audio_opt:* if 0: 不推送音频* if 1: 推送编码前音频(PCM)* if 2: 推送编码后音频(aac/pcma/pcmu/speex).* * @param video_opt:* if 0: 不推送视频* if 1: 推送编码前视频(NV12/I420/RGBA8888等格式)* if 2: 推送编码后视频(AVC/HEVC)* if 3: 层叠加模式** <pre>This function must be called firstly.</pre>** @return the handle of publisher instance*/public native long SmartPublisherOpen(Object ctx, int audio_opt, int video_opt,  int width, int height);/*** 设置流类型* @param type: 0:表示 live 流, 1:表示 on-demand 流, SDK默认为0(live流)* 注意: 流类型设置当前仅对GB28181媒体流有效* @return {0} if successful*/public native int SetStreamType(long handle, int type);/*** 投递视频 on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer** @param codec_id: 编码id, 当前支持H264和H265, 1:H264, 2:H265** @param packet: 视频数据, 包格式请参考H264/H265 Annex B Byte stream format, 例如:*                0x00000001 nal_unit 0x00000001 ...*                H264 IDR: 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....*                H265 IDR: 0x00000001 vps 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....** @param offset: 偏移量* @param size: packet size* @param pts_us: 时间戳, 单位微秒* @param is_pts_discontinuity: 是否时间戳间断,0:未间断,1:间断* @param is_key: 是否是关键帧, 0:非关键帧, 1:关键帧* @param codec_specific_data: 可选参数,可传null, 对于H264关键帧包, 如果packet不含sps和pps, 可传0x00000001 sps 0x00000001 pps*                    ,对于H265关键帧包, 如果packet不含vps,sps和pps, 可传0x00000001 vps 0x00000001 sps 0x00000001 pps* @param codec_specific_data_size: codec_specific_data size* @param width: 图像宽, 可传0* @param height: 图像高, 可传0** @return {0} if successful*/public native int PostVideoOnDemandPacketByteBuffer(long handle, int codec_id,ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity, int is_key,byte[] codec_specific_data, int codec_specific_data_size,int width, int height);/*** 投递音频on demand包, 当前只用于GB28181推送, 注意ByteBuffer对象必须是DirectBuffer** @param codec_id: 编码id, 当前支持PCMA和AAC, 65536:PCMA, 65538:AAC* @param packet: 音频数据* @param offset:packet偏移量* @param size: packet size* @param pts_us: 时间戳, 单位微秒* @param is_pts_discontinuity: 是否时间戳间断,0:未间断,1:间断* @param codec_specific_data: 如果是AAC的话,需要传 Audio Specific Configuration* @param codec_specific_data_size: codec_specific_data size* @param sample_rate: 采样率* @param channels: 通道数** @return {0} if successful*/public native int PostAudioOnDemandPacketByteBuffer(long handle, int codec_id,ByteBuffer packet, int offset, int size, long pts_us, int is_pts_discontinuity,byte[] codec_specific_data, int codec_specific_data_size,int sample_rate, int channels);/*** on demand source完成seek后, 请调用* @return {0} if successful*/public native int OnSeekProcessed(long handle);/*** 启动 GB28181 媒体流** @return {0} if successful*/public native int StartGB28181MediaStream(long handle);/*** 停止 GB28181 媒体流** @return {0} if successful*/public native int StopGB28181MediaStream(long handle);/*** 关闭推送实例,结束时必须调用close接口释放资源** @return {0} if successful*/public native int SmartPublisherClose(long handle);}/**
* Listener部分实现代码
*/public class PlaybackListenerImpl implements com.gb.ntsignalling.GBSIPAgentPlaybackListener {/**收到s=Playback的文件下载Invite*/@Overridepublic void ntsOnInvitePlayback(long id, String deviceId, SessionDescription sdp) {if (!post_task(new PlaybackListenerImpl.OnInviteTask(this.context_, this.is_exit_, this.senders_map_, deviceId, sdp, id))) {Log.e(TAG, "ntsOnInvitePlayback post_task failed, " + RecordSender.make_print_tuple(id, deviceId, sdp.getTime().getStartTime(),  sdp.getTime().getStopTime()));// 这里不发488, 等待事务超时也可以的GBSIPAgent agent = this.context_.get_agent();if (agent != null)agent.respondPlaybackInvite(488, id, deviceId);}}/**发送Playback invite response 异常*/@Overridepublic void ntsOnPlaybackInviteResponseException(long id, String deviceId, int statusCode, String errorInfo) {Log.i(TAG, "ntsOnPlaybackInviteResponseException, status_code:" + statusCode + ", "+ RecordSender.make_print_tuple(id, deviceId) + ",  error_info:" + errorInfo);RecordSender sender = senders_map_.remove(id);if (null == sender)return;PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);if (!post_task(task))task.run();}/** 收到CANCEL Playback INVITE请求*/@Overridepublic void ntsOnCancelPlayback(long id, String deviceId) {Log.i(TAG, "ntsOnCancelPlayback, " + RecordSender.make_print_tuple(id, deviceId));RecordSender sender = senders_map_.remove(id);if (null == sender)return;PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);if (!post_task(task))task.run();}/** 收到Ack*/@Overridepublic void ntsOnAckPlayback(long id, String deviceId) {Log.i(TAG, "ntsOnAckPlayback, "+ RecordSender.make_print_tuple(id, deviceId));RecordSender sender = senders_map_.get(id);if (null == sender) {Log.e(TAG, "ntsOnAckPlayback get sender is null, " + RecordSender.make_print_tuple(id, deviceId));GBSIPAgent agent = this.context_.get_agent();if (agent != null)agent.terminatePlayback(id, deviceId, false);return;}PlaybackListenerImpl.StartTask task = new PlaybackListenerImpl.StartTask(sender, this.senders_map_);if (!post_task(task))task.run();}/** 收到Bye*/@Overridepublic void ntsOnByePlayback(long id, String deviceId) {Log.i(TAG, "ntsOnByePlayback, "+ RecordSender.make_print_tuple(id, deviceId));RecordSender sender = this.senders_map_.remove(id);if (null == sender)return;PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);if (!post_task(task))task.run();}/** 播放命令*/@Overridepublic void ntsOnPlaybackMANSRTSPPlayCommand(long id, String deviceId) {RecordSender sender = this.senders_map_.get(id);if (null == sender) {Log.e(TAG, "ntsOnPlaybackMANSRTSPPlayCommand can not get sender " + RecordSender.make_print_tuple(id, deviceId));return;}sender.post_play_command();Log.i(TAG, "ntsOnPlaybackMANSRTSPPlayCommand " + RecordSender.make_print_tuple(id, deviceId));}/** 暂停命令*/@Overridepublic void ntsOnPlaybackMANSRTSPPauseCommand(long id, String deviceId) {RecordSender sender = this.senders_map_.get(id);if (null == sender) {Log.e(TAG, "ntsOnPlaybackMANSRTSPPauseCommand can not get sender " + RecordSender.make_print_tuple(id, deviceId));return;}sender.post_pause_command();Log.i(TAG, "ntsOnPlaybackMANSRTSPPauseCommand " + RecordSender.make_print_tuple(id, deviceId));}/** 快进/慢进命令*/@Overridepublic void ntsOnPlaybackMANSRTSPScaleCommand(long id, String deviceId, double scale) {if (scale < 0.01) {Log.e(TAG, "ntsOnPlaybackMANSRTSPScaleCommand invalid scale:" + scale  + " " + RecordSender.make_print_tuple(id, deviceId));return;}RecordSender sender = this.senders_map_.get(id);if (null == sender) {Log.e(TAG, "ntsOnPlaybackMANSRTSPScaleCommand can not get sender, scale:" + scale  + " " + RecordSender.make_print_tuple(id, deviceId));return;}sender.post_scale_command(scale);Log.i(TAG, "ntsOnPlaybackMANSRTSPScaleCommand, scale:" + scale + " " + RecordSender.make_print_tuple(id, deviceId));}/** 随机拖动命令*/@Overridepublic void ntsOnPlaybackMANSRTSPSeekCommand(long id, String device_id, double position_sec) {if (position_sec < 0.0) {Log.e(TAG, "ntsOnPlaybackMANSRTSPSeekCommand invalid seek pos:" + position_sec  + ", " + RecordSender.make_print_tuple(id, device_id));return;}RecordSender sender = this.senders_map_.get(id);if (null == sender) {Log.e(TAG, "ntsOnPlaybackMANSRTSPSeekCommand can not get sender " + RecordSender.make_print_tuple(id, device_id));return;}long offset_ms = sender.get_file_start_time_offset_ms();position_sec += (offset_ms/1000.0);sender.post_seek_command(position_sec);Log.i(TAG, "ntsOnPlaybackMANSRTSPSeekCommand seek pos:" + RecordSender.out_point_3(position_sec) + "s, " + RecordSender.make_print_tuple(id, device_id));}/** 停止命令*/@Overridepublic void ntsOnPlaybackMANSRTSPTeardownCommand(long id, String device_id) {CallTerminatePlaybackTask call_terminate_task =  new CallTerminatePlaybackTask(this.context_, id, device_id, true);post_task(call_terminate_task);RecordSender sender = this.senders_map_.remove(id);if (null == sender) {Log.w(TAG, "ntsOnPlaybackMANSRTSPTeardownCommand can not remove sender " + RecordSender.make_print_tuple(id, device_id));return;}Log.i(TAG, "ntsOnPlaybackMANSRTSPTeardownCommand " + RecordSender.make_print_tuple(id, device_id));PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);if (!post_task(task))task.run();}/** 不是在收到BYE Message情况下, 终止Playback*/@Overridepublic void ntsOnTerminatePlayback(long id, String deviceId) {Log.i(TAG, "ntsOnTerminatePlayback, "+ RecordSender.make_print_tuple(id, deviceId));RecordSender sender = this.senders_map_.remove(id);if (null == sender)return;PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);if (!post_task(task))task.run();}/** Playback会话对应的对话终止, 一般不会触发这个回调,目前只有在响应了200K, 但在64*T1时间后还没收到ACK,才可能会出发收到这个, 请做相关清理处理*/@Overridepublic void ntsOnPlaybackDialogTerminated(long id, String deviceId) {Log.i(TAG, "ntsOnPlaybackDialogTerminated, "+ RecordSender.make_print_tuple(id, deviceId));RecordSender sender = this.senders_map_.remove(id);if (null == sender)return;PlaybackListenerImpl.StopDisposeTask task = new PlaybackListenerImpl.StopDisposeTask(sender);if (!post_task(task))task.run();}
}
支持云台控制和预置位查询
@Override
public void ntsOnDevicePresetQueryCommand(String fromUserName, String fromUserNameAtDomain, String sn, String deviceId) {handler_.postDelayed(new Runnable() {@Overridepublic void run() {Log.i(TAG, "DaniuSDK ntsOnDevicePresetQueryCommand from_user_name:" + from_user_name_ + ", sn:" + sn_ + ", device_id:" + device_id_);List<com.gb.ntsignalling.PresetItem> preset_list = new LinkedList<>();preset_list.add(new com.gb.ntsignalling.PresetItem("1", "Android PreSet1"));preset_list.add(new com.gb.ntsignalling.PresetItem("2", "Android PreSet2"));if (gb28181_agent_ != null )gb28181_agent_.respondDevicePresetQueryCommand(this.from_user_name_, this.from_user_name_at_domain_, this.sn_, this.device_id_, preset_list);}private String from_user_name_;private String from_user_name_at_domain_;private String sn_;private String device_id_;public Runnable set(String from_user_name, String from_user_name_at_domain,String sn, String device_id) {this.from_user_name_ = from_user_name;this.from_user_name_at_domain_ = from_user_name_at_domain;this.sn_ = sn;this.device_id_ = device_id;return this;}}.set(fromUserName, fromUserNameAtDomain, sn, deviceId),0);
}@Override
public void ntsOnDeviceControlPTZCmd(String deviceId, String typeValue) {handler_.postDelayed(new Runnable() {@Overridepublic void run() {Log.i(TAG, "DaniuSDK ntsOnDeviceControlPTZCmd device_id:" + device_id_ + " PTZType:" + ptz_type_);if (null == ptz_type_)return;ptz_type_ = ptz_type_.trim();if (ptz_type_.length() != 16)return;int instruction = hexStringToInt(ptz_type_.substring(6, 8));int combination_code2 = hexStringToInt(ptz_type_.substring(12, 14));//Android平台GB28181设备接入端,针对性的解析处理即可,这里不再赘述private String device_id_;private String ptz_type_;public Runnable set(String device_id, String ptz_type) {this.device_id_ = device_id;this.ptz_type_ = ptz_type;return this;}}.set(deviceId, typeValue),0);}
[实时水印]支持动态文字水印、png水印
watermarkSelctor = (Spinner) findViewById(R.id.watermarkSelctor);final String[] watermarks = new String[]{"图片水印", "全部水印", "文字水印", "不加水印"};ArrayAdapter<String> adapterWatermark = new ArrayAdapter<String>(this,android.R.layout.simple_spinner_item, watermarks);adapterWatermark.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);watermarkSelctor.setAdapter(adapterWatermark);watermarkSelctor.setSelection(3,true);
watemarkType = 3;   //默认不加水印watermarkSelctor.setOnItemSelectedListener(new AdapterView.OnItemSelectedListener() {@Overridepublic void onItemSelected(AdapterView<?> parent, View view,int position, long id) {watemarkType = position;Log.i(TAG, "[水印类型]Currently choosing: " + watermarks[position] + ", watemarkType: " + watemarkType);if (layer_post_thread_ != null) {layer_post_thread_.enableText(isHasTextWatermark());layer_post_thread_.enablePicture(isHasPictureWatermark());}}@Overridepublic void onNothingSelected(AdapterView<?> parent) {}
});
[实时快照]支持实时快照
class ButtonCaptureImageListener implements View.OnClickListener {@SuppressLint("SimpleDateFormat")public void onClick(View v) {if(isPushingRtmp || isRecording || isRTSPPublisherRunning || isGB28181StreamRunning){String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());String imageFileName = "dn_" + timeStamp;    //创建以时间命名的文件名称String imagePath = imageSavePath + "/" + imageFileName + ".png";Log.i(TAG, "imagePath:" + imagePath);libPublisher.SmartPublisherSaveCurImage(publisherHandle, imagePath);}else{Log.e(TAG, "快照失败,请确保在推送、录像或内置RTSP服务发布状态..");}}
}
[降噪]支持环境音、手机干扰等引起的噪音降噪处理、自动增益、VAD检测
boolean is_noise_suppression = true;
libPublisher.SmartPublisherSetNoiseSuppression(publisherHandle, is_noise_suppression ? 1 : 0);boolean is_agc = false;
libPublisher.SmartPublisherSetAGC(publisherHandle, is_agc ? 1 : 0);int echo_cancel_delay = 0;
libPublisher.SmartPublisherSetEchoCancellation(publisherHandle, 1, echo_cancel_delay);
外部编码前后视频数据对接
  • 编码前数据(目前支持的有YV12/NV21/NV12/I420/RGB24/RGBA32/RGB565等数据类型),其中,Android平台前后摄像头数据,或者屏幕数据,或者Unity拿到的数据,均属编码前数据;
  • 编码后数据(如无人机等264/HEVC数据,或者本地解析的MP4音视频数据);
 [扩展录像功能]支持和录像SDK组合使用,录像相关功能
class ButtonStartRecorderListener implements View.OnClickListener {public void onClick(View v) {if (isRecording) {stopRecorder();if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {ConfigControlEnable(true);}btnStartRecorder.setText("实时录像");btnPauseRecorder.setText("暂停录像");btnPauseRecorder.setEnabled(false);isPauseRecording = true;return;}Log.i(TAG, "onClick start recorder..");if (libPublisher == null)return;if (!isPushingRtmp && !isRTSPPublisherRunning&& !isGB28181StreamRunning) {InitAndSetConfig();}ConfigRecorderParam();int startRet = libPublisher.SmartPublisherStartRecorder(publisherHandle);if (startRet != 0) {if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {if (publisherHandle != 0) {long handle = publisherHandle;publisherHandle = 0;libPublisher.SmartPublisherClose(handle);}}Log.e(TAG, "Failed to start recorder.");return;}if (!isPushingRtmp && !isRTSPPublisherRunning && !isGB28181StreamRunning) {CheckInitAudioRecorder();ConfigControlEnable(false);}startLayerPostThread();btnStartRecorder.setText("停止录像");isRecording = true;btnPauseRecorder.setEnabled(true);isPauseRecording = true;}
}class ButtonPauseRecorderListener implements View.OnClickListener {public void onClick(View v) {if (isRecording) {if(isPauseRecording){int ret = libPublisher.SmartPublisherPauseRecorder(publisherHandle, 1);if (ret == 0){isPauseRecording = false;btnPauseRecorder.setText("恢复录像");}else if(ret == 3){Log.e(TAG, "Pause recorder failed, please re-try again..");}else{Log.e(TAG, "Pause recorder failed..");}}else{int ret = libPublisher.SmartPublisherPauseRecorder(publisherHandle, 0);if (ret == 0){isPauseRecording = true;btnPauseRecorder.setText("暂停录像");}else if(ret == 3){Log.e(TAG, "Resume recorder failed, please re-try again..");}else{Log.e(TAG, "Resume recorder failed..");}}}}
}

总结

Android平台GB28181设备接入侧模块,如果需要做的更好,上述提到的技术层面的问题解决了还不够,还需要针对各类国标平台适配对接,只有这样,才能更好的为执法记录仪、智能安全帽、智能监控、智慧零售、智慧教育、远程办公、明厨亮灶、智慧交通、智慧工地、雪亮工程、平安乡村、生产运输、车载终端等场景服务。

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.mzph.cn/news/171120.shtml

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈email:809451989@qq.com,一经查实,立即删除!

相关文章

Scannet v2 数据集介绍以及子集下载展示

Scannet v2 数据集介绍以及子集下载展示 文章目录 Scannet v2 数据集介绍以及子集下载展示参考数据集简介子集scannet_frames_25kscannet_frames_test 下载脚本 download_scannetv2.py 参考 scannet数据集简介和下载-CSDN博客 scannet v2 数据集下载_scannetv2数据集_蓝羽飞鸟的…

BeanUtil的正确使用方式

shigen日更文章的博客写手&#xff0c;擅长Java、python、vue、shell等编程语言和各种应用程序、脚本的开发。记录成长&#xff0c;分享认知&#xff0c;留住感动。 在实际的开发中&#xff0c;我们常常会用到工具类去拷贝对象的属性&#xff0c;将一个对象的属性转换成另外一个…

球面的表面积

此推导需要用到重积分的知识&#xff0c;另外关于曲面的面积公式可以看我之前的博客

百度AI布局:从财报看百度的核心竞争力和未来发展方向

百度是中国最大的搜索引擎&#xff0c;也是全球领先的人工智能&#xff08;AI&#xff09;公司。百度在2023年第三季度业绩中&#xff0c;展示了其在AI领域的强劲表现和广阔前景。 百度财报透露了关于AI业务的哪些重要信息&#xff1f; 百度在2023年第三季度的财报中&#xf…

机器学习:攻击方法FGSM系列

任务 FGSM I-FGSM MI-FGSM Ensemble Attack 攻击评价指标 准确率越低表明攻击越好 数据 预训练模型 BaseLine 实践

***Linux常用命令及解释

1、查看Linux的版本信息 1.1、uname -a 1.2、cat /etc/issue 1.3、cat /proc/version 1.4、hostnamectl 通过使用hostnamectl命令&#xff0c;可以查询和更改系统主机名&#xff0c;并且还可以查看Linux的发行版和内核版本。 2、删除文件 3、修改目录权限 4、解压文件 5、…

探索 Linux vim/vi 编辑器:介绍、模式以及基本操作演示

&#x1f490;作者&#xff1a;insist-- &#x1f490;个人主页&#xff1a;insist-- 的个人主页 理想主义的花&#xff0c;最终会盛开在浪漫主义的土壤里&#xff0c;我们的热情永远不会熄灭&#xff0c;在现实平凡中&#xff0c;我们终将上岸&#xff0c;阳光万里 ❤️欢迎点…

Spring的依赖注入,依赖注入的基本原则,依赖注入的优势

文章目录 Spring的依赖注入依赖注入的基本原则依赖注入有什么优势查找定位操作与应用代码完全无关。有哪些不同类型的依赖注入实现方式&#xff1f;构造器依赖注入和 Setter方法注入的区别 Spring的依赖注入 控制反转IoC是一个很大的概念&#xff0c;可以用不同的方式来实现。…

android系统新特性——用户界面以及系统界面改进

用户界面改进 Android用户界面改进最明显的就是MD了。MD是Google于2014年推出的设计语言&#xff0c;它是一套完整的设计系统&#xff0c;包含了动画、样式、布局、组件等一系列与设计有关的元素。通过对这些行为的描述&#xff0c;让开发者设计出更符合目标的软件&#xff0c…

vue3+elementPlus之侧边菜单栏功能

选择默认的颜色&#xff0c;将代码拷贝至<el-aside>模块中 稍微把不需要的修改一下。 <template><div class"common-layout"><el-container><el-header class"homeHeader"><div class"headerTitle">Devops…

vue怎么实现国际化? vue-i18n 插件实现国际化,支持切换不同语言

依赖的文档开始 | Vue I18n 一、安装 npm install vue-i18n 如果在一个模块系统中使用它&#xff0c;你必须通过 Vue.use() 明确地安装 vue-i18n&#xff1a; import Vue from vue import VueI18n from vue-i18nVue.use(VueI18n)二、使用 在 src 下创建 lang 文件夹 1.准…

实现【Linux--NTP 时间同步服务搭建】

实现【Linux--NTP 时间同步服务搭建】 &#x1f53b; 前言&#x1f53b; 一、NTP 校时&#x1f530; 1.1 NTP 服务校时与 ntpdate 校时的区别&#x1f530; 1.2 NTP 校时服务搭建&#x1f530; 1.2.1 确认 ntp 的安装&#x1f530; 1.2.2 配置 ntp 服务&#x1f530; 1.2.3 启动…

大数据面试大厂真题【附答案详细解析】

1.Java基础篇&#xff08;阿里、蚂蚁、字节、携程、快手、杭州银行等&#xff09; 问题&#xff1a;HashMap的底层实现原理 答案&#xff1a; 在jdk1.8之前&#xff0c;hashmap由 数组-链表数据结构组成&#xff0c;在jdk1.8之后hashmap由 数组-链表-红黑树数据结构组成&…

openpnp - 给底部相机加防尘罩

文章目录 openpnp - 给底部相机加防尘罩概述笔记END openpnp - 给底部相机加防尘罩 概述 设备标定完, 看着底部相机, 有点担心掉进去东西, 万一从吸嘴掉下去的料(或者清理设备台面时, 不小心掉进去东西)将顶部相机搞短路怎么办. 就想加个防尘罩, 如果有东西掉进去, 可以掉到机…

makefile 学习(5)完整的makefile模板

参考自&#xff1a; (1&#xff09;深度学习部署笔记(二): g, makefile语法&#xff0c;makefile自己的CUDA编程模板(2&#xff09;https://zhuanlan.zhihu.com/p/396448133(3) 一个挺好的工程模板&#xff0c;(https://github.com/shouxieai/cpp-proj-template) 1. c 编译流…

开源vs闭源,处在大模型洪流中,向何处去?

文章目录 一、开源和闭源的优劣势比较1.1 开源优势1.2 闭源的优势 二、开源和闭源对大模型技术发展的影响2.1 数据共享2.2 算法创新2.3 业务拓展2.4 安全性和隐私2.5 社会责任和伦理 三、开源与闭源的商业模式比较3.1 盈利模式3.2 市场竞争3.3 用户生态3.4 创新速度 四&#xf…

shiro的前后端分离模式

shiro的前后端分离模式 前言&#xff1a;在上一篇《shiro的简单认证和授权》中介绍了shiro的搭建&#xff0c;默认情况下&#xff0c;shiro是通过设置cookie&#xff0c;使前端请求带有“JSESSION”cookie&#xff0c;后端通过获取该cookie判断用户是否登录以及授权。但是在前…

TCP/IP协议、三次握手、四次挥手

TCP/IP TCP/IP协议分层TCP头部三次握手TCP四次挥手常见问题1、什么是TCP网络分层2、TCP为什么是三次握手&#xff0c;不是两次或者四次&#xff1f;3、TCP为什么是四次挥手&#xff0c;为什么不能是三次挥手将第二次挥手和第三次挥手合并&#xff1f;4、四次挥手时为什么TIME_W…

ZKP11.4 Use CI to instantiate Fiat-Shamir

ZKP学习笔记 ZK-Learning MOOC课程笔记 Lecture 11: From Practice to Theory (Guest Lecturer: Alex Lombardi) 11.4 Use CI to instantiate Fiat-Shamir Avoid Bad Challenges Def: Given false claim x x x and a first message α \alpha α, a challenge β \beta …

再见 Pandas,再见算法

大家好,《再见pandas》 系列已有200多位朋友加入学习了,这段时间亲眼见证了很多朋友的飞跃进步,从无到有,从一个问问题的小白到开始慢慢回答别人的问题,在讨论和练习中不断成长。虽说pandas已经很普及了,但普及内容的深度却远远不够。 下面这套原创图文是我和几位小伙伴…