一、简介
Audio是多媒体子系统中的一个重要模块，其波及的内容比拟多，有音频的渲染、音频的采集、音频的策略管理等。本文次要针对音频渲染性能进行具体地剖析，并通过源码中提供的例子，对音频渲染进行流程的梳理。
二、目录foundation/multimedia/audio_framework
audio_framework├── frameworks│ ├── js #js 接口│ │ └── napi│ │ └── audio_renderer #audio_renderer NAPI接口│ │ ├── include│ │ │ ├── audio_renderer_callback_napi.h│ │ │ ├── renderer_data_request_callback_napi.h│ │ │ ├── renderer_period_position_callback_napi.h│ │ │ └── renderer_position_callback_napi.h│ │ └── src│ │ ├── audio_renderer_callback_napi.cpp│ │ ├── audio_renderer_napi.cpp│ │ ├── renderer_data_request_callback_napi.cpp│ │ ├── renderer_period_position_callback_napi.cpp│ │ └── renderer_position_callback_napi.cpp│ └── native #native 接口│ └── audiorenderer│ ├── BUILD.gn│ ├── include│ │ ├── audio_renderer_private.h│ │ └── audio_renderer_proxy_obj.h│ ├── src│ │ ├── audio_renderer.cpp│ │ └── audio_renderer_proxy_obj.cpp│ └── test│ └── example│ └── audio_renderer_test.cpp├── interfaces│ ├── inner_api #native实现的接口│ │ └── native│ │ └── audiorenderer #audio渲染本地实现的接口定义│ │ └── include│ │ └── audio_renderer.h│ └── kits #js调用的接口│ └── js│ └── audio_renderer #audio渲染NAPI接口的定义│ └── include│ └── audio_renderer_napi.h└── services #服务端 └── audio_service ├── BUILD.gn ├── client #IPC调用中的proxy端 │ ├── include │ │ ├── audio_manager_proxy.h │ │ ├── audio_service_client.h │ └── src │ ├── audio_manager_proxy.cpp │ ├── audio_service_client.cpp └── server #IPC调用中的server端 ├── include │ └── audio_server.h └── src ├── audio_manager_stub.cpp └── audio_server.cpp三、音频渲染总体流程
四、Native接口应用
在OpenAtom OpenHarmony（以下简称“OpenHarmony”）零碎中，音频模块提供了性能测试代码，本文选取了其中的音频渲染例子作为切入点来进行介绍，例子采纳的是对wav格局的音频文件进行渲染。wav格局的音频文件是wav头文件和音频的原始数据，不须要进行数据解码，所以音频渲染间接对原始数据进行操作，文件门路为：foundation/multimedia/audio_framework/frameworks/native/audiorenderer/test/example/audio_renderer_test.cpp
bool TestPlayback(int argc, char *argv[]) const{ FILE* wavFile = fopen(path, "rb"); //读取wav文件头信息 size_t bytesRead = fread(&wavHeader, 1, headerSize, wavFile); //设置AudioRenderer参数 AudioRendererOptions rendererOptions = {}; rendererOptions.streamInfo.encoding = AudioEncodingType::ENCODING_PCM; rendererOptions.streamInfo.samplingRate = static_cast<AudioSamplingRate>(wavHeader.SamplesPerSec); rendererOptions.streamInfo.format = GetSampleFormat(wavHeader.bitsPerSample); rendererOptions.streamInfo.channels = static_cast<AudioChannel>(wavHeader.NumOfChan); rendererOptions.rendererInfo.contentType = contentType; rendererOptions.rendererInfo.streamUsage = streamUsage; rendererOptions.rendererInfo.rendererFlags = 0; //创立AudioRender实例 unique_ptr<AudioRenderer> audioRenderer = AudioRenderer::Create(rendererOptions); shared_ptr<AudioRendererCallback> cb1 = make_shared<AudioRendererCallbackTestImpl>(); //设置音频渲染回调 ret = audioRenderer->SetRendererCallback(cb1); //InitRender办法次要调用了audioRenderer实例的Start办法，启动音频渲染 if (!InitRender(audioRenderer)) { AUDIO_ERR_LOG("AudioRendererTest: Init render failed"); fclose(wavFile); return false; } //StartRender办法次要是读取wavFile文件的数据，而后通过调用audioRenderer实例的Write办法进行播放 if (!StartRender(audioRenderer, wavFile)) { AUDIO_ERR_LOG("AudioRendererTest: Start render failed"); fclose(wavFile); return false; } //进行渲染 if (!audioRenderer->Stop()) { AUDIO_ERR_LOG("AudioRendererTest: Stop failed"); } //开释渲染 if (!audioRenderer->Release()) { AUDIO_ERR_LOG("AudioRendererTest: Release failed"); } //敞开wavFile fclose(wavFile); return true; }首先读取wav文件，通过读取到wav文件的头信息对AudioRendererOptions相干的参数进行设置，包含编码格局、采样率、采样格局、通道数等。依据AudioRendererOptions设置的参数来创立AudioRenderer实例（实际上是AudioRendererPrivate），后续的音频渲染次要是通过AudioRenderer实例进行。创立实现后，调用AudioRenderer的Start办法，启动音频渲染。启动后，通过AudioRenderer实例的Write办法，将数据写入，音频数据会被播放。
五、调用流程
- 创立AudioRenderer
std::unique_ptr<AudioRenderer> AudioRenderer::Create(const std::string cachePath, const AudioRendererOptions &rendererOptions, const AppInfo &appInfo){ ContentType contentType = rendererOptions.rendererInfo.contentType; StreamUsage streamUsage = rendererOptions.rendererInfo.streamUsage; AudioStreamType audioStreamType = AudioStream::GetStreamType(contentType, streamUsage); auto audioRenderer = std::make_unique<AudioRendererPrivate>(audioStreamType, appInfo); if (!cachePath.empty()) { AUDIO_DEBUG_LOG("Set application cache path"); audioRenderer->SetApplicationCachePath(cachePath); } audioRenderer->rendererInfo_.contentType = contentType; audioRenderer->rendererInfo_.streamUsage = streamUsage; audioRenderer->rendererInfo_.rendererFlags = rendererOptions.rendererInfo.rendererFlags; AudioRendererParams params; params.sampleFormat = rendererOptions.streamInfo.format; params.sampleRate = rendererOptions.streamInfo.samplingRate; params.channelCount = rendererOptions.streamInfo.channels; params.encodingType = rendererOptions.streamInfo.encoding; if (audioRenderer->SetParams(params) != SUCCESS) { AUDIO_ERR_LOG("SetParams failed in renderer"); audioRenderer = nullptr; return nullptr; } return audioRenderer;}首先通过AudioStream的GetStreamType办法获取音频流的类型，依据音频流类型创立AudioRendererPrivate对象，AudioRendererPrivate是AudioRenderer的子类。紧接着对audioRenderer进行参数设置，其中包含采样格局、采样率、通道数、编码格局。设置实现后返回创立的AudioRendererPrivate实例。
- 设置回调
int32_t AudioRendererPrivate::SetRendererCallback(const std::shared_ptr<AudioRendererCallback> &callback){ RendererState state = GetStatus(); if (state == RENDERER_NEW || state == RENDERER_RELEASED) { return ERR_ILLEGAL_STATE; } if (callback == nullptr) { return ERR_INVALID_PARAM; } // Save reference for interrupt callback if (audioInterruptCallback_ == nullptr) { return ERROR; } std::shared_ptr<AudioInterruptCallbackImpl> cbInterrupt = std::static_pointer_cast<AudioInterruptCallbackImpl>(audioInterruptCallback_); cbInterrupt->SaveCallback(callback); // Save and Set reference for stream callback. Order is important here. if (audioStreamCallback_ == nullptr) { audioStreamCallback_ = std::make_shared<AudioStreamCallbackRenderer>(); if (audioStreamCallback_ == nullptr) { return ERROR; } } std::shared_ptr<AudioStreamCallbackRenderer> cbStream =std::static_pointer_cast<AudioStreamCallbackRenderer>(audioStreamCallback_); cbStream->SaveCallback(callback); (void)audioStream_->SetStreamCallback(audioStreamCallback_); return SUCCESS;}参数传入的回调次要波及到两个方面：一方面是AudioInterruptCallbackImpl中设置了咱们传入的渲染回调，另一方面是AudioStreamCallbackRenderer中也设置了渲染回调。
- 启动渲染
bool AudioRendererPrivate::Start(StateChangeCmdType cmdType) const{ AUDIO_INFO_LOG("AudioRenderer::Start"); RendererState state = GetStatus(); AudioInterrupt audioInterrupt; switch (mode_) { case InterruptMode::SHARE_MODE: audioInterrupt = sharedInterrupt_; break; case InterruptMode::INDEPENDENT_MODE: audioInterrupt = audioInterrupt_; break; default: break; } AUDIO_INFO_LOG("AudioRenderer::Start::interruptMode: %{public}d, streamType: %{public}d, sessionID: %{public}d", mode_, audioInterrupt.streamType, audioInterrupt.sessionID); if (audioInterrupt.streamType == STREAM_DEFAULT || audioInterrupt.sessionID == INVALID_SESSION_ID) { return false; } int32_t ret = AudioPolicyManager::GetInstance().ActivateAudioInterrupt(audioInterrupt); if (ret != 0) { AUDIO_ERR_LOG("AudioRendererPrivate::ActivateAudioInterrupt Failed"); return false; } return audioStream_->StartAudioStream(cmdType);}AudioPolicyManager::GetInstance().ActivateAudioInterrupt这个操作次要是依据AudioInterrupt来进行音频中断的激活，这里波及了音频策略相干的内容，后续会专门出对于音频策略的文章进行剖析。这个办法的外围是通过调用AudioStream的StartAudioStream办法来启动音频流。
bool AudioStream::StartAudioStream(StateChangeCmdType cmdType){ int32_t ret = StartStream(cmdType); resetTime_ = true; int32_t retCode = clock_gettime(CLOCK_MONOTONIC, &baseTimestamp_); if (renderMode_ == RENDER_MODE_CALLBACK) { isReadyToWrite_ = true; writeThread_ = std::make_unique<std::thread>(&AudioStream::WriteCbTheadLoop, this); } else if (captureMode_ == CAPTURE_MODE_CALLBACK) { isReadyToRead_ = true; readThread_ = std::make_unique<std::thread>(&AudioStream::ReadCbThreadLoop, this); } isFirstRead_ = true; isFirstWrite_ = true; state_ = RUNNING; AUDIO_INFO_LOG("StartAudioStream SUCCESS"); if (audioStreamTracker_) { AUDIO_DEBUG_LOG("AudioStream:Calling Update tracker for Running"); audioStreamTracker_->UpdateTracker(sessionId_, state_, rendererInfo_, capturerInfo_); } return true;}AudioStream的StartAudioStream次要的工作是调用StartStream办法，StartStream办法是AudioServiceClient类中的办法。AudioServiceClient类是AudioStream的父类。接下来看一下AudioServiceClient的StartStream办法。
int32_t AudioServiceClient::StartStream(StateChangeCmdType cmdType){ int error; lock_guard<mutex> lockdata(dataMutex); pa_operation *operation = nullptr; pa_threaded_mainloop_lock(mainLoop); pa_stream_state_t state = pa_stream_get_state(paStream); streamCmdStatus = 0; stateChangeCmdType_ = cmdType; operation = pa_stream_cork(paStream, 0, PAStreamStartSuccessCb, (void *)this); while (pa_operation_get_state(operation) == PA_OPERATION_RUNNING) { pa_threaded_mainloop_wait(mainLoop); } pa_operation_unref(operation); pa_threaded_mainloop_unlock(mainLoop); if (!streamCmdStatus) { AUDIO_ERR_LOG("Stream Start Failed"); ResetPAAudioClient(); return AUDIO_CLIENT_START_STREAM_ERR; } else { AUDIO_INFO_LOG("Stream Started Successfully"); return AUDIO_CLIENT_SUCCESS; }}StartStream办法中次要是调用了pulseaudio库的pa_stream_cork办法进行流启动，后续就调用到了pulseaudio库中了。pulseaudio库咱们暂且不剖析。
- 写入数据
int32_t AudioRendererPrivate::Write(uint8_t *buffer, size_t bufferSize){ return audioStream_->Write(buffer, bufferSize);}通过调用AudioStream的Write形式实现性能，接下来看一下AudioStream的Write办法。
size_t AudioStream::Write(uint8_t *buffer, size_t buffer_size){ int32_t writeError; StreamBuffer stream; stream.buffer = buffer; stream.bufferLen = buffer_size; isWriteInProgress_ = true; if (isFirstWrite_) { if (RenderPrebuf(stream.bufferLen)) { return ERR_WRITE_FAILED; } isFirstWrite_ = false; } size_t bytesWritten = WriteStream(stream, writeError); isWriteInProgress_ = false; if (writeError != 0) { AUDIO_ERR_LOG("WriteStream fail,writeError:%{public}d", writeError); return ERR_WRITE_FAILED; } return bytesWritten;}Write办法中分成两个阶段，首次写数据，先调用RenderPrebuf办法，将preBuf_的数据写入后再调用WriteStream进行音频数据的写入。
size_t AudioServiceClient::WriteStream(const StreamBuffer &stream, int32_t &pError){ size_t cachedLen = WriteToAudioCache(stream); if (!acache.isFull) { pError = error; return cachedLen; } pa_threaded_mainloop_lock(mainLoop); const uint8_t *buffer = acache.buffer.get(); size_t length = acache.totalCacheSize; error = PaWriteStream(buffer, length); acache.readIndex += acache.totalCacheSize; acache.isFull = false; if (!error && (length >= 0) && !acache.isFull) { uint8_t *cacheBuffer = acache.buffer.get(); uint32_t offset = acache.readIndex; uint32_t size = (acache.writeIndex - acache.readIndex); if (size > 0) { if (memcpy_s(cacheBuffer, acache.totalCacheSize, cacheBuffer + offset, size)) { AUDIO_ERR_LOG("Update cache failed"); pa_threaded_mainloop_unlock(mainLoop); pError = AUDIO_CLIENT_WRITE_STREAM_ERR; return cachedLen; } AUDIO_INFO_LOG("rearranging the audio cache"); } acache.readIndex = 0; acache.writeIndex = 0; if (cachedLen < stream.bufferLen) { StreamBuffer str; str.buffer = stream.buffer + cachedLen; str.bufferLen = stream.bufferLen - cachedLen; AUDIO_DEBUG_LOG("writing pending data to audio cache: %{public}d", str.bufferLen); cachedLen += WriteToAudioCache(str); } } pa_threaded_mainloop_unlock(mainLoop); pError = error; return cachedLen;}WriteStream办法不是间接调用pulseaudio库的写入办法，而是通过WriteToAudioCache办法将数据写入缓存中，如果缓存没有写满则间接返回，不会进入上面的流程，只有当缓存写满后，才会调用上面的PaWriteStream办法。该办法波及对pulseaudio库写入操作的调用，所以缓存的目标是防止对pulseaudio库频繁地做IO操作，进步了效率。
六、总结
本文次要对OpenHarmony 3.2 Beta多媒体子系统的音频渲染模块进行介绍，首先梳理了Audio Render的整体流程，而后对几个外围的办法进行代码的剖析。整体的流程次要通过pulseaudio库启动流，而后通过pulseaudio库的pa_stream_write办法进行数据的写入，最初播放出音频数据。
音频渲染次要分为以下几个档次：
（1）AudioRenderer的创立，理论创立的是它的子类AudioRendererPrivate实例。
（2）通过AudioRendererPrivate设置渲染的回调。
（3）启动渲染，这一部分代码最终会调用到pulseaudio库中，相当于启动了pulseaudio的流。
（4）通过pulseaudio库的pa_stream_write办法将数据写入设施，进行播放。
对OpenHarmony 3.2 Beta多媒体系列开发感兴趣的读者，也能够浏览我之前写过几篇文章：
《OpenHarmony 3.2 Beta多媒体系列——视频录制》
《OpenHarmony 3.2 Beta源码剖析之MediaLibrary》
《OpenHarmony 3.2 Beta多媒体系列——音视频播放框架》
《OpenHarmony 3.2 Beta多媒体系列——音视频播放gstreamer》。