前言

产品一直有用户反馈音频截断问题。在机遇巧合下现学现卖音频知识处理相关问题。

问题描述

我们查看以下简化播放器代码：

class AACPlayer(private val filePath: String) {private val TAG = "AACPlayer"private var extractor: MediaExtractor? = nullprivate var codec: MediaCodec? = nullprivate var audioTrack: AudioTrack? = nullfun play() {try {extractor = MediaExtractor().apply {setDataSource(filePath)}var trackIndex = -1for (i in 0 until extractor!!.trackCount) {val format = extractor!!.getTrackFormat(i)val mime = format.getString(MediaFormat.KEY_MIME)if (mime!!.startsWith("audio/")) {trackIndex = ibreak}}if (trackIndex >= 0) {extractor!!.selectTrack(trackIndex)val format = extractor!!.getTrackFormat(trackIndex)val mime = format.getString(MediaFormat.KEY_MIME)codec = MediaCodec.createDecoderByType(mime!!)codec!!.configure(format, null, null, 0)codec!!.start()val sampleRate = format.getInteger(MediaFormat.KEY_SAMPLE_RATE)val channelConfig =if (format.getInteger(MediaFormat.KEY_CHANNEL_COUNT) == 1) AudioFormat.CHANNEL_OUT_MONO else AudioFormat.CHANNEL_OUT_STEREOval bufferSize = AudioTrack.getMinBufferSize(sampleRate,channelConfig,AudioFormat.ENCODING_PCM_16BIT);val audioTrackAttributes =AudioAttributes.Builder().setUsage(AudioAttributes.USAGE_MEDIA).setContentType(AudioAttributes.CONTENT_TYPE_MUSIC).build()val audioFormat = AudioFormat.Builder().setEncoding(AudioFormat.ENCODING_PCM_16BIT).setSampleRate(sampleRate).setChannelMask(channelConfig).build()audioTrack = AudioTrack.Builder().setAudioAttributes(audioTrackAttributes).setAudioFormat(audioFormat).setTransferMode(AudioTrack.MODE_STREAM).setBufferSizeInBytes(bufferSize).build()if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {logD("bufferSizeInFrames = [${audioTrack?.bufferSizeInFrames}] bufferCapacityInFrames = [${audioTrack?.bufferCapacityInFrames}] bufferSize = [${bufferSize}]  startThresholdInFrames = [${audioTrack!!.startThresholdInFrames}]")}audioTrack!!.play()val inputBuffers = codec!!.inputBuffersval outputBuffers = codec!!.outputBuffersval bufferInfo = MediaCodec.BufferInfo()var isEOS = falsewhile (!isEOS) {val inIndex = codec!!.dequeueInputBuffer(10000)if (inIndex >= 0) {val buffer = inputBuffers[inIndex]val sampleSize = extractor!!.readSampleData(buffer, 0)if (sampleSize < 0) {codec!!.queueInputBuffer(inIndex,0,0,0,MediaCodec.BUFFER_FLAG_END_OF_STREAM)isEOS = true} else {val presentationTimeUs = extractor!!.sampleTimecodec!!.queueInputBuffer(inIndex, 0, sampleSize, presentationTimeUs, 0)extractor!!.advance()}}var outIndex = codec!!.dequeueOutputBuffer(bufferInfo, 10000)while (outIndex >= 0) {val outBuffer = outputBuffers[outIndex]val bufferBackup = outBuffer.slice()if (outBuffer.remaining() <= 0) {continue}//仅仅为了打印无他用val array = ByteArray(bufferBackup.remaining())bufferBackup.get(array, 0, array.size)logD(array.joinToString(transform = { String.format("%02x", it) }))logD("写入数据大小${array.size} hashCode ${array.contentHashCode()}")audioTrack!!.write(outBuffer,outBuffer.remaining(),AudioTrack.WRITE_BLOCKING)codec!!.releaseOutputBuffer(outIndex, false)outIndex = codec!!.dequeueOutputBuffer(bufferInfo, 0)}}}} catch (e: Exception) {e.printStackTrace()} finally {extractor?.release()codec?.stop()codec?.release()audioTrack?.flush()audioTrack?.stop()audioTrack?.release()}}fun logD(msg:String) {Log.d(TAG, msg)}
}

这也是网上充斥最多的示例代码，但是上面的代码丢失尾帧的音频的问题。

getStartThresholdInFrames文档

在Android中audiotrack有一个缓冲区，调用则可以阻塞或阻塞式使用audiotrack.write向里面写入数据。播放器为提高效率在缓冲大于startThresholdInFrames时取出进行播放。startThresholdInFrames一般大于等于bufferSizeInFrames。

你播放音频时不敢保证所有音频数据都是对齐startThresholdInFrames，所以你会以为调用audiotrack.flush可以解决问题了。但是我们阅读相关文档flush文档发现这个API只是丢弃之前的数据，加速audiotrack.write。

解决方案 在音频流写入结束调用audiotrack.stop 这个函数会将未播放的数据进行加载播放在结束。audiotrack.stop文档

在上述的代码我们如下编写：

class AACPlayer(...) {//...fun play() {try {while (outIndex >= 0) {//....略//结束调用stop刷出残余音频if (bufferInfo.size == 0&& (bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {audioTrack!!.stop()break}//....略}}catch(...){//...}finally{} finally {extractor?.release()codec?.stop()codec?.release()audioTrack?.flush()//注释多余的stop//audioTrack?.stop()audioTrack?.release()}}
}

当然你如果比较骚可以进行补帧操作

class AACPlayer(...) {//...fun play() {try {while (outIndex >= 0) {//....略if (bufferInfo.size == 0&& (bufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {var interpolationFrame =(audioTrack!!.startThresholdInFrames / bufferSize) + 1while (interpolationFrame > 0) {interpolationFrame--;audioTrack!!.write(ByteArray(bufferSize), 0, bufferSize)}break}//....略}}catch(...){//...}finally{} finally {//略}}
}

实践

我们有一个极短音频且有效音在末尾，那么在部分手机上将无法听到这个音频

在某手机上相关输出参数如下：

bufferSizeInFrames = [11310] 
bufferCapacityInFrames = [11310] 
bufferSize = [45240]  
startThresholdInFrames = [11310]

这个文件对应的PCM数据40960(A000h)字节大小。
我们看下这个文件的末端可以看到很多有效数据。

我们在看看文件最前面PCM数据 全是空数据。

所以这个文件只有末尾才音频。
我们算一下Audiotrack刷新次数

文件PCM大小/Audiotrack刷新阈值 =40960/11310 = 3.6

假设如果我们有效音频在最后0.6将无法播放