显示成果
背景
在利用的开发过程中，咱们防止不了要应用 Opengl 共享 context 的技术。比方咱们须要在预览的同时还要将预览的画面传递给 MediaCodec 进行编码。有的敌人会说 Camera2 能够设置多个 surface，这样就能够把预览的 surface 和 MediaCodec 的 surface 同时传递给 Camera2。这种只是实现比较简单的录制性能。当初少数有录制性能的利用都反对滤镜性能，能够把你拍得更丑陋。因为退出了滤镜的性能，所以咱们要通过 Opengl 技术来实现。
当咱们退出 opengl 的滤镜性能后就有一个须要解决的问题，如何让预览和 MediaCo dec 同时利用滤镜。这时共享 context 技术就派上用场了，通过共享 context 使创立在不同线程的 texture id 能够互相共享。通常共享的 texture id 绑定的是一个 frame buffer，滤镜成果都绘制在这个 frame buffer 上，而后将 frame buffer 别离绘制在预览和 MediaCodec。这里不对共享 context 的实现进行具体的介绍，这里重点关注的是有没有其余的计划来实现这个性能。
对于 Camera2 的思考（SurfaceTextureRenderer）
大家都晓得 Camera2 是反对设置多个 surface 的，那么他是如何把 camera 数据绘制到这些 surface 上的呢？实现形式就在 SurfaceTextureRenderer 中，通过查看这个文件咱们理解到他结构了一个 opengl 环境用于绘制 camera 数据。在构建 opengl 环境的过程中创立了 EGLDisplay、EGLContext、EGLSurface。这里对于 EGLSurface 的创立有些非凡，咱们能够看到代码中创立了多个 EGLSurface。每个 EGLSurface 都与一个 surface 进行绑定。
private void configureEGLOutputSurfaces(Collection<EGLSurfaceHolder> surfaces) {if (surfaces == null || surfaces.size() == 0) {throw new IllegalStateException("No Surfaces were provided to draw to");
}
int[] surfaceAttribs = {EGL14.EGL_NONE};
for (EGLSurfaceHolder holder : surfaces) {
holder.eglSurface = EGL14.eglCreateWindowSurface(mEGLDisplay, mConfigs,
holder.surface, surfaceAttribs, /*offset*/ 0);
checkEglError("eglCreateWindowSurface");
}
}上面的代码是绘制 camera 数据到各个 EGLSurface 上，具体绘制到哪个 EGLSurface 上是通过 makeCurrent 办法管制的。看到这里大家大略就明确了，通过 EGL14.eglMakeCurrent 办法就能够实现绘制内容到不同的 surface 上。当然在切换 surface 的时候要注意绘制的 viewport 大小也会发送变动，所以 mvp matrix 也要进行相应的更新。
public void drawIntoSurfaces(CaptureCollector targetCollector) {
...
for (EGLSurfaceHolder holder : mSurfaces) {if (LegacyCameraDevice.containsSurfaceId(holder.surface, targetSurfaceIds)) {
try{
LegacyCameraDevice.setSurfaceDimens(holder.surface, holder.width,
holder.height);
makeCurrent(holder.eglSurface);
LegacyCameraDevice.setNextTimestamp(holder.surface, captureHolder.second);
drawFrame(mSurfaceTexture, holder.width, holder.height,
(mFacing == CameraCharacteristics.LENS_FACING_FRONT) ?
FLIP_TYPE_HORIZONTAL : FLIP_TYPE_NONE);
swapBuffers(holder.eglSurface);
} catch (LegacyExceptionUtils.BufferQueueAbandonedException e) {Log.w(TAG, "Surface abandoned, dropping frame.", e);
request.setOutputAbandoned();}
}
}
...
}
}
private void makeCurrent(EGLSurface surface)
throws LegacyExceptionUtils.BufferQueueAbandonedException {EGL14.eglMakeCurrent(mEGLDisplay, surface, surface, mEGLContext);
checkEglDrawError("makeCurrent");
}通过增加两个 SurfaceView 验证计划
这里咱们创立两个 surfaceview 用于验证结构多个 eglsurface 的计划。
surfaceView1.holder.addCallback(object:SurfaceHolder.Callback{override fun surfaceCreated(holder: SurfaceHolder?) { }
override fun surfaceChanged(holder: SurfaceHolder?, format: Int, width: Int, height: Int) {if (eglSurfaceHolder1 == null) {eglSurfaceHolder1 = EGLCore.EGLSurfaceHolder(holder!!.surface, width.toFloat(), height.toFloat())
renderScope.addSurfaceHolder(eglSurfaceHolder1)
}
}
override fun surfaceDestroyed(holder: SurfaceHolder?) {renderScope.removeSurfaceHolder(eglSurfaceHolder1)
eglSurfaceHolder1 = null
}
})
surfaceView2.holder.addCallback(object:SurfaceHolder.Callback{override fun surfaceCreated(holder: SurfaceHolder?) { }
override fun surfaceChanged(holder: SurfaceHolder?, format: Int, width: Int, height: Int) {if (eglSurfaceHolder2 == null) {eglSurfaceHolder2 = EGLCore.EGLSurfaceHolder(holder!!.surface, width.toFloat(), height.toFloat())
renderScope.addSurfaceHolder(eglSurfaceHolder2)
}
}
override fun surfaceDestroyed(holder: SurfaceHolder?) {renderScope.removeSurfaceHolder(eglSurfaceHolder2)
eglSurfaceHolder2 = null
}
})这里把两个 surface 都增加到 RenderScope 中，RenderScope 创立 opengl 线程环境，opengl 描绘都是产生在这个线程的。
class RenderScope(private val render: Render) : BackgroundScope by HandlerThreadScope() {
private var eglCore: EGLCore? = null
init {
runInBackground {eglCore = EGLCore()
render.onCreate()}
}
fun addSurfaceHolder(eglSurfaceHolder: EGLCore.EGLSurfaceHolder?) = runInBackground {eglCore?.addSurfaceHolder(eglSurfaceHolder)
}
fun removeSurfaceHolder(eglSurfaceHolder: EGLCore.EGLSurfaceHolder?) = runInBackground {eglCore?.removeSurfaceHolder(eglSurfaceHolder)
}
fun removeSurfaceHolder(surface: Surface?) = runInBackground {eglCore?.removeSurfaceHolder(surface)
}
fun release() = runInBackground {render.onDestroy()
eglCore?.releaseEGLContext()
eglCore = null
quit()}
fun requestRender() = runInBackground {eglCore?.render { render.onDrawFrame(it) }
}
interface Render {fun onCreate()
fun onDestroy()
fun onDrawFrame(eglSurfaceHolder: EGLCore.EGLSurfaceHolder)
}
}EGLCore 这个类是用来结构 opengl 环境的，结构的过程这里不细说了。咱们重点关注下 addSurfaceHolder 这个办法。它是用于增加 surface 的，通过这个办法增加 surface 后，渲染过程中就能够把内容绘制到这个 surface 上。render 办法用于渲染应用，看他的实现次要就是通过 makecurrent 来切换 eglsurface，而后调用 block 进行描绘。block 调用的次数与增加的 surface 数量雷同。
class EGLCore {
private var mEGLDisplay = EGL14.EGL_NO_DISPLAY
private var mEGLContext = EGL14.EGL_NO_CONTEXT
private var mConfigs: EGLConfig? = null
private val mEGLSurfaces = SparseArray<EGLSurfaceHolder>()
init {log("initEGLContext")
releaseEGLContext()
mEGLDisplay = EGL14.eglGetDisplay(EGL14.EGL_DEFAULT_DISPLAY)
check(!(mEGLDisplay === EGL14.EGL_NO_DISPLAY)) {"No EGL14 display"}
val version = IntArray(2)
check(EGL14.eglInitialize(mEGLDisplay, version, /*offset*/0, version, /*offset*/1)) {"Cannot initialize EGL14"}
val attributeList = intArrayOf(
EGL14.EGL_RED_SIZE, EGL_COLOR_BIT_LENGTH,
EGL14.EGL_GREEN_SIZE, EGL_COLOR_BIT_LENGTH,
EGL14.EGL_BLUE_SIZE, EGL_COLOR_BIT_LENGTH,
EGL14.EGL_RENDERABLE_TYPE, EGL14.EGL_OPENGL_ES2_BIT,
EGL_RECORDABLE_ANDROID, 1,
EGL14.EGL_SURFACE_TYPE, EGL14.EGL_PBUFFER_BIT or EGL14.EGL_WINDOW_BIT,
EGL14.EGL_NONE
)
val configs = arrayOfNulls<EGLConfig>(1)
val numConfigs = IntArray(1)
EGL14.eglChooseConfig(
mEGLDisplay,
attributeList, /*offset*/0,
configs, /*offset*/0, configs.size,
numConfigs, /*offset*/0
)
checkEglError("eglCreateContext RGB888+recordable ES2")
mConfigs = configs[0]
val contextAttributeList = intArrayOf(
EGL14.EGL_CONTEXT_CLIENT_VERSION, GLES_VERSION,
EGL14.EGL_NONE
)
mEGLContext = EGL14.eglCreateContext(
mEGLDisplay,
configs[0],
EGL14.EGL_NO_CONTEXT,
contextAttributeList, /*offset*/0
)
checkEglError("eglCreateContext")
check(!(mEGLContext === EGL14.EGL_NO_CONTEXT)) {"No EGLContext could be made"}
EGL14.eglMakeCurrent(mEGLDisplay, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE, mEGLContext)
}
fun addSurfaceHolder(eglSurfaceHolder: EGLSurfaceHolder?) {if (mEGLDisplay == EGL14.EGL_NO_DISPLAY) {log("addSurfaceHolder mEGLDisplay == EGL14.EGL_NO_DISPLAY")
return
}
eglSurfaceHolder ?: return
check(eglSurfaceHolder.surface.isValid) {"addSurface surface is not valid!!"}
val surfaceAttribute = intArrayOf(EGL14.EGL_NONE)
eglSurfaceHolder.eglSurface = EGL14.eglCreateWindowSurface(
mEGLDisplay,
mConfigs,
eglSurfaceHolder.surface,
surfaceAttribute, /*offset*/0
)
checkEglError("eglCreateWindowSurface")
mEGLSurfaces.put(eglSurfaceHolder.surface.hashCode(), eglSurfaceHolder)
}
fun removeSurfaceHolder(eglSurfaceHolder: EGLSurfaceHolder?) {if (mEGLDisplay == EGL14.EGL_NO_DISPLAY) {log("removeSurfaceHolder mEGLDisplay == EGL14.EGL_NO_DISPLAY")
return
}
eglSurfaceHolder ?: return
mEGLSurfaces[eglSurfaceHolder.surface.hashCode()]?.let {EGL14.eglDestroySurface(mEGLDisplay, it.eglSurface)
it.eglSurface = null
mEGLSurfaces.remove(it.surface.hashCode())
}
}
fun removeSurfaceHolder(surface: Surface?) {if (mEGLDisplay == EGL14.EGL_NO_DISPLAY) {log("removeSurfaceHolder mEGLDisplay == EGL14.EGL_NO_DISPLAY")
return
}
surface ?: return
mEGLSurfaces[surface.hashCode()]?.let {EGL14.eglDestroySurface(mEGLDisplay, it.eglSurface)
it.eglSurface = null
mEGLSurfaces.remove(surface.hashCode())
}
}
fun render(block: (EGLSurfaceHolder) -> Unit) {if (mEGLDisplay == EGL14.EGL_NO_DISPLAY) {log("render mEGLDisplay == EGL14.EGL_NO_DISPLAY")
return
}
mEGLSurfaces.forEach { _, holder ->
if (!holder.available) {return@forEach}
EGL14.eglMakeCurrent(mEGLDisplay, holder.eglSurface, holder.eglSurface, mEGLContext)
checkEglError("makeCurrent")
block.invoke(holder)
val result = EGL14.eglSwapBuffers(mEGLDisplay, holder.eglSurface)
when (val error = EGL14.eglGetError()) {
EGL14.EGL_SUCCESS -> result
EGL14.EGL_BAD_NATIVE_WINDOW, EGL14.EGL_BAD_SURFACE -> throw IllegalStateException("swapBuffers: EGL error: 0x" + Integer.toHexString(error)
)
else -> throw IllegalStateException("swapBuffers: EGL error: 0x" + Integer.toHexString(error)
)
}
}
}
fun releaseEGLContext() {if (mEGLDisplay !== EGL14.EGL_NO_DISPLAY) {log("releaseEGLContext")
EGL14.eglMakeCurrent(mEGLDisplay, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_CONTEXT)
mEGLSurfaces.forEach { _, holder ->
if (holder.eglSurface != null) {EGL14.eglDestroySurface(mEGLDisplay, holder.eglSurface)
holder.eglSurface = null
}
}
mEGLSurfaces.clear()
EGL14.eglDestroyContext(mEGLDisplay, mEGLContext)
EGL14.eglReleaseThread()
EGL14.eglTerminate(mEGLDisplay)
}
mConfigs = null
mEGLDisplay = EGL14.EGL_NO_DISPLAY
mEGLContext = EGL14.EGL_NO_CONTEXT
}
private fun checkEglError(msg: String) {val error = EGL14.eglGetError()
check(error == EGL14.EGL_SUCCESS) {msg + ": EGL error: 0x" + Integer.toHexString(error) }
}
private fun log(msg: String) {Log.d("EGLCore", msg)
}
class EGLSurfaceHolder(
val surface: Surface,
val width: Float,
val height: Float
) {
var eglSurface: EGLSurface? = null
var available = true
val mvpMatrix = MVPMatrix().updateViewport(width, height)
val viewPortRectF = RectF(0f, 0f, width, height)
}
companion object {
private const val GLES_VERSION = 2
private const val EGL_COLOR_BIT_LENGTH = 8
const val EGL_RECORDABLE_ANDROID = 0x3142 // from EGL/eglext.h
}
}Git
https://github.com/mjlong1231…