关于android:Android-鼠标光标的图形合成

始终很好奇鼠标光标是如何实现的,它反映很快、提早很小,没有受到 Android 显示零碎的影响。正好最近做相干的工作,跟着源码好好钻研一下。

本文参考 Android 9.0 源码。

从 Input 说起

咱们并不是要讲 Input,只想看看鼠标光标的绘制过程。然而,Android 将鼠标光标的实现放到了 Input 中,这看起来也是正当的。在 Input 中,光标由类Sprite 实现。源码中对 Sprite 的解释为:显示在其余图层之上的图形对象。看来 Sprite 并非专为光标设计,但在源码中的地位表明,它在 Android 中也只为鼠标或触摸之类的输出设施的光标服务。Sprite 的定义中也只提供了简略的图形操作。

frameworks/base/libs/input/SpriteController.h

/*
 * A sprite is a simple graphical object that is displayed on-screen above other layers.
 * The basic sprite class is an interface.
 * The implementation is provided by the sprite controller.
 */
class Sprite : public RefBase {
protected:
    Sprite() { }
    virtual ~Sprite() { }

public:
    enum {
        // The base layer for pointer sprites.
        BASE_LAYER_POINTER = 0, // reserve space for 1 pointer

        // The base layer for spot sprites.
        BASE_LAYER_SPOT = 1, // reserve space for MAX_POINTER_ID spots
    };

    /* Sets the bitmap that is drawn by the sprite.
     * The sprite retains a copy of the bitmap for subsequent rendering. */
    virtual void setIcon(const SpriteIcon& icon) = 0;

    inline void clearIcon() {
        setIcon(SpriteIcon());
    }

    /* Sets whether the sprite is visible. */
    virtual void setVisible(bool visible) = 0;

    /* Sets the sprite position on screen, relative to the sprite's hot spot. */
    virtual void setPosition(float x, float y) = 0;

    /* Sets the layer of the sprite, relative to the system sprite overlay layer.
     * Layer 0 is the overlay layer, > 0 appear above this layer. */
    virtual void setLayer(int32_t layer) = 0;

    /* Sets the sprite alpha blend ratio between 0.0 and 1.0. */
    virtual void setAlpha(float alpha) = 0;

    /* Sets the sprite transformation matrix. */
    virtual void setTransformationMatrix(const SpriteTransformationMatrix& matrix) = 0;
};

管制光标的类叫做 SpriteController,PointerController 会应用这个类来显示光标。这里咱们只关怀光标图形的合成,真正显示和更新光标的办法是 SpriteController::doUpdateSprites()

frameworks/base/libs/input/SpriteController.cpp

void SpriteController::doUpdateSprites() {
    // 从invalidatedSprites 中收集须要更新的 Sprite
    Vector<SpriteUpdate> updates;
    size_t numSprites;
    { // acquire lock
        AutoMutex _l(mLock);

        numSprites = mLocked.invalidatedSprites.size();
        for (size_t i = 0; i < numSprites; i++) {
            const sp<SpriteImpl>& sprite = mLocked.invalidatedSprites.itemAt(i);

            updates.push(SpriteUpdate(sprite, sprite->getStateLocked()));
            sprite->resetDirtyLocked();
        }
        mLocked.invalidatedSprites.clear();
    } // release lock

    // surfaces 未创立或失落时,从新创立 surface
    bool surfaceChanged = false;
    for (size_t i = 0; i < numSprites; i++) {
        SpriteUpdate& update = updates.editItemAt(i);

        if (update.state.surfaceControl == NULL && update.state.wantSurfaceVisible()) {
            update.state.surfaceWidth = update.state.icon.bitmap.width();
            update.state.surfaceHeight = update.state.icon.bitmap.height();
            update.state.surfaceDrawn = false;
            update.state.surfaceVisible = false;
            // 创立 Surface,咱们这次的关注点
            update.state.surfaceControl = obtainSurface(
                    update.state.surfaceWidth, update.state.surfaceHeight);
            if (update.state.surfaceControl != NULL) {
                update.surfaceChanged = surfaceChanged = true;
            }
        }
    }

    // 如果须要,从新调整 sprites 大小
    SurfaceComposerClient::Transaction t;
    bool needApplyTransaction = false;
    for (size_t i = 0; i < numSprites; i++) {
        ......
            if (update.state.surfaceWidth < desiredWidth
                    || update.state.surfaceHeight < desiredHeight) {
                needApplyTransaction = true;

                t.setSize(update.state.surfaceControl,
                        desiredWidth, desiredHeight);
                ......
            }
        }
    }
    if (needApplyTransaction) {
        t.apply();
    }

    // 如果须要,重画 sprites
    for (size_t i = 0; i < numSprites; i++) {
        SpriteUpdate& update = updates.editItemAt(i);

        if ((update.state.dirty & DIRTY_BITMAP) && update.state.surfaceDrawn) {
            update.state.surfaceDrawn = false;
            update.surfaceChanged = surfaceChanged = true;
        }

        if (update.state.surfaceControl != NULL && !update.state.surfaceDrawn
                && update.state.wantSurfaceVisible()) {
            sp<Surface> surface = update.state.surfaceControl->getSurface();
            ANativeWindow_Buffer outBuffer;
            ......
                // 应用 SKIA 画图
                SkBitmap surfaceBitmap;
                ssize_t bpr = outBuffer.stride * bytesPerPixel(outBuffer.format);
                surfaceBitmap.installPixels(SkImageInfo::MakeN32Premul(outBuffer.width, outBuffer.height),
                                            outBuffer.bits, bpr);

                SkCanvas surfaceCanvas(surfaceBitmap);

                SkPaint paint;
                paint.setBlendMode(SkBlendMode::kSrc);
                surfaceCanvas.drawBitmap(update.state.icon.bitmap, 0, 0, &paint);

                if (outBuffer.width > update.state.icon.bitmap.width()) {
                    paint.setColor(0); // transparent fill color
                    surfaceCanvas.drawRect(SkRect::MakeLTRB(update.state.icon.bitmap.width(), 0,
                            outBuffer.width, update.state.icon.bitmap.height()), paint);
                }
                if (outBuffer.height > update.state.icon.bitmap.height()) {
                    paint.setColor(0); // transparent fill color
                    surfaceCanvas.drawRect(SkRect::MakeLTRB(0, update.state.icon.bitmap.height(),
                            outBuffer.width, outBuffer.height), paint);
                }
                ......
    }

    // 依据 dirty 值来设置 Surface
    needApplyTransaction = false;
    for (size_t i = 0; i < numSprites; i++) {
        SpriteUpdate& update = updates.editItemAt(i);

        bool wantSurfaceVisibleAndDrawn = update.state.wantSurfaceVisible()
                && update.state.surfaceDrawn;
        bool becomingVisible = wantSurfaceVisibleAndDrawn && !update.state.surfaceVisible;
        bool becomingHidden = !wantSurfaceVisibleAndDrawn && update.state.surfaceVisible;
        if (update.state.surfaceControl != NULL && (becomingVisible || becomingHidden
                || (wantSurfaceVisibleAndDrawn && (update.state.dirty & (DIRTY_ALPHA
                        | DIRTY_POSITION | DIRTY_TRANSFORMATION_MATRIX | DIRTY_LAYER
                        | DIRTY_VISIBILITY | DIRTY_HOTSPOT))))) {
        ......
    }

    if (needApplyTransaction) {
        status_t status = t.apply();
        if (status) {
            ALOGE("Error applying Surface transaction");
        }
    }
......
}

一次的光标的更新就会波及到如此多的代码逻辑,可见UI真是不容易。其余的逻辑线不论,这次咱们只关怀光标的图层。上述代码通过 obtainSurface() 来创立 Surface。

frameworks/base/libs/input/SpriteController.cpp

sp<SurfaceControl> SpriteController::obtainSurface(int32_t width, int32_t height) {
    ensureSurfaceComposerClient();

    sp<SurfaceControl> surfaceControl = mSurfaceComposerClient->createSurface(
            String8("Sprite"), width, height, PIXEL_FORMAT_RGBA_8888,
            ISurfaceComposerClient::eHidden |
            ISurfaceComposerClient::eCursorWindow);
    if (surfaceControl == NULL || !surfaceControl->isValid()) {
        ALOGE("Error creating sprite surface.");
        return NULL;
    }
    return surfaceControl;
}

这里咱们须要重点关注的是 createSurface() 办法中的参数 flags。Sprite 中这个 flags 设置了eHiddeneCursorWindow,它们表明创立的 Surface 是暗藏的,并标识为 Cursor 应用。

来到 Surface

Input 中为光标创立了一个 Surface,并且标识这是一个 Cursor 应用的 Surface。之后,Surface 中会依据情景对光标图层做非凡解决,这里的关键字就是 Cursor

咱们还是以光标图层为主线进行跟踪,先持续看下createSurface()。通过一系列的 Binder 调用和 Message传递,最终通过 SurfaceFlinger 的createLayer()实现图层创立。

frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

status_t SurfaceFlinger::createLayer(const String8& name, const sp<Client>& client, uint32_t w,
                                     uint32_t h, PixelFormat format, uint32_t flags,
                                     int32_t windowType, int32_t ownerUid, sp<IBinder>* handle,
                                     sp<IGraphicBufferProducer>* gbp,
                                     const sp<IBinder>& parentHandle,
                                     const sp<Layer>& parentLayer) {
    ......
    switch (flags & ISurfaceComposerClient::eFXSurfaceMask) {
        // 一般图层
        case ISurfaceComposerClient::eFXSurfaceNormal:
            result = createBufferLayer(client,
                    uniqueName, w, h, flags, format,
                    handle, gbp, &layer);

            break;
        // 纯色图层
        case ISurfaceComposerClient::eFXSurfaceColor:
            result = createColorLayer(client,
                    uniqueName, w, h, flags,
                    handle, &layer);
            break;
        default:
            result = BAD_VALUE;
            break;
    }
    ......
    // Client中通过Layer治理Surface,将创立的Layer退出到LayerStack中
    result = addClientLayer(client, *handle, *gbp, layer, parentHandle, parentLayer);
    if (result != NO_ERROR) {
        return result;
    }
    mInterceptor->saveSurfaceCreation(layer);

    setTransactionFlags(eTransactionNeeded);
    return result;
}

createLayer()中,光标算是一般图层,所以仅需调用createBufferLayer()来创立。

frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

status_t SurfaceFlinger::createBufferLayer(const sp<Client>& client,
        const String8& name, uint32_t w, uint32_t h, uint32_t flags, PixelFormat& format,
        sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp, sp<Layer>* outLayer)
{
    ......
    // 创立一个BufferLayer
    sp<BufferLayer> layer = new BufferLayer(this, client, name, w, h, flags);
    // 设置Buffer属性
    status_t err = layer->setBuffers(w, h, format, flags);
    if (err == NO_ERROR) {
        *handle = layer->getHandle(); // 获取Layer的句柄
        *gbp = layer->getProducer(); // 获取GraphicBufferProducer对象 
        *outLayer = layer;
    }

    ALOGE_IF(err, "createBufferLayer() failed (%s)", strerror(-err));
    return err;
}

其中layer->setBuffers()设置了该BufferLayer的属性。能够看到,当申请的是一个 Cursor 图层时,mPotentialCursor被设置为true,表明该 BufferLayer 作为 Cursor 应用。

frameworks/native/services/surfaceflinger/BufferLayer.cpp

status_t BufferLayer::setBuffers(uint32_t w, uint32_t h, PixelFormat format, uint32_t flags) {
    ......
    mFormat = format;

    mPotentialCursor = (flags & ISurfaceComposerClient::eCursorWindow) ? true : false;
    mProtectedByApp = (flags & ISurfaceComposerClient::eProtectedByApp) ? true : false;
    mCurrentOpacity = getOpacityForFormat(format);

    mConsumer->setDefaultBufferSize(w, h);
    mConsumer->setDefaultBufferFormat(format);
    mConsumer->setConsumerUsageBits(getEffectiveUsage(0));

    return NO_ERROR;
}

SurfaceFlinger 中的 Cursor 操作

下面讲到 Cursor Layer 最外围的属性mPotentialCursorcreateSurface()只是设置了这个属性,真正的应用在 SurfaceFlinger 渲染过程中。接着我发现,想把这个货色看明确,先须要把 Android 图形合成弄清楚,这可是的宏大的工程。借张图,有趣味的本人钻研。

然而,工夫无限,怎么办?我的解决办法就是搜寻关键字。搜寻关键字Cursor后,能够失去一些相干的操作。SurfaceFlinger 接管到 Vsync 信号后,会调用handleMessageRefresh()来刷新显示。

frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

void SurfaceFlinger::handleMessageRefresh() {
    ......
    preComposition(refreshStartTime); //合成预处理
    rebuildLayerStacks(); //从新构建LayerStacks
    setUpHWComposer(); //更新HWComposer的图层和属性
    doDebugFlashRegions(); //图形绘制的debug模式
    doTracing("handleRefresh");
    logLayerStats();
    doComposition(); //合成所有图层
    postComposition(refreshStartTime); //合成后处理
    ......
}

咱们还是只关怀 Cursor 的操作,它位于 HWComposer 管制的图层中。

frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

void SurfaceFlinger::setUpHWComposer() {
    ......
    // 遍历所有的DisplayDevice,为绘制做筹备
    for (size_t dpy=0 ; dpy<mDisplays.size() ; dpy++) {
        ......
        mDisplays[dpy]->beginFrame(mustRecompose);

        if (mustRecompose) {
            mDisplays[dpy]->lastCompositionHadVisibleLayers = !empty;
        }
    }

    // 设置HWC Layer
    if (CC_UNLIKELY(mGeometryInvalid)) {
        mGeometryInvalid = false;
        for (size_t dpy=0 ; dpy<mDisplays.size() ; dpy++) {
            ......
                for (size_t i = 0; i < currentLayers.size(); i++) {
                    const auto& layer = currentLayers[i];
                    // 尝试创立HWC Layer,如果失败则强制OpenGL渲染
                    if (!layer->hasHwcLayer(hwcId)) {
                        if (!layer->createHwcLayer(getBE().mHwc.get(), hwcId)) {
                            layer->forceClientComposition(hwcId);
                            continue;
                        }
                    }

                    // 设置HWC Layer的显示区域、合成模式、Alpha、Order等
                    layer->setGeometry(displayDevice, i);
                    // HWC被禁止或绘制debug模式时,强制OpenGL渲染
                    if (mDebugDisableHWC || mDebugRegion) {
                        layer->forceClientComposition(hwcId);
                    }
        ......
    }

    // 筹备HWC须要渲染的数据
    for (size_t displayId = 0; displayId < mDisplays.size(); ++displayId) {
        auto& displayDevice = mDisplays[displayId];
        const auto hwcId = displayDevice->getHwcDisplayId();
        ......
            //调用 setPerFrameData办法
            layer->setPerFrameData(displayDevice);
        ......
    }
    ......
    for (size_t displayId = 0; displayId < mDisplays.size(); ++displayId) {
        ......
        // 尝试进行显示
        status_t result = displayDevice->prepareFrame(*getBE().mHwc);
        ......
    }
}

其中setPerFrameData()实现 HWComposer 的相干设置,为显示做筹备。

frameworks/native/services/surfaceflinger/BufferLayer.cpp

void BufferLayer::setPerFrameData(const sp<const DisplayDevice>& displayDevice) {
    ......
    // 设置可见区域
    auto error = hwcLayer->setVisibleRegion(visible);
    ......
    // 设置刷新区域
    error = hwcLayer->setSurfaceDamage(surfaceDamageRegion);
    ......
    // Sideband layers设置
    if (getBE().compositionInfo.hwc.sidebandStream.get()) {
        setCompositionType(hwcId, HWC2::Composition::Sideband);
        error = hwcLayer->setSidebandStream(getBE().compositionInfo.hwc.sidebandStream->handle());
        ......
        return;
    }

    if (mPotentialCursor) {
        // Cursor layers设置
        setCompositionType(hwcId, HWC2::Composition::Cursor);
    } else {
        // Device layers设置
        setCompositionType(hwcId, HWC2::Composition::Device);
    }

    // 设置色调空间
    error = hwcLayer->setDataspace(mCurrentDataSpace);
    if (error != HWC2::Error::None) {
        ALOGE("[%s] Failed to set dataspace %d: %s (%d)", mName.string(), mCurrentDataSpace,
              to_string(error).c_str(), static_cast<int32_t>(error));
    }

    // 获取HDR数据并设置到HWC中
    const HdrMetadata& metadata = mConsumer->getCurrentHdrMetadata();
    error = hwcLayer->setPerFrameMetadata(displayDevice->getSupportedPerFrameMetadata(), metadata);
    ......
    // 获取渲染的数据buffer和Fence,设置到HWC中
    sp<GraphicBuffer> hwcBuffer;
    hwcInfo.bufferCache.getHwcBuffer(getBE().compositionInfo.mBufferSlot,
                                     getBE().compositionInfo.mBuffer, &hwcSlot, &hwcBuffer);

    auto acquireFence = mConsumer->getCurrentFence();
    error = hwcLayer->setBuffer(hwcSlot, hwcBuffer, acquireFence);
    ......
}

咱们终于找到了心愿看到的mPotentialCursor,通过这个标识通知 HWC2 这是一个 CursorLayer。除此之外,对于 CursorLayer 的操作与 DeviceLayer 并没有区别。所以,SurfaceFlinger 更多的是心愿 HWComposer 依据 Layer 的类型进行不同解决。目前 HWC2 反对的 Layer 类型有,

  1. HWC2_COMPOSITION_CLIENT:不通过 HWC 硬件来合成图层。GPU 将这类图层合成到一个图像 Buffer 中,而后传递给 HWC。
  2. HWC2_COMPOSITION_DEVICE:应用 HWC 硬件来合成图层。
  3. HWC2_COMPOSITION_SOLID_COLOR:用来解决 ColorLayer 数据,如果 HWC 不反对,则改为应用 CLIENT 形式合成。
  4. HWC2_COMPOSITION_CURSOR:用来解决 CursorLayer 数据,地位通过setCursorPosition 异步设置。如果 HWC 不反对,则改为应用 CLIENT 或 DEVICE 形式合成。
  5. HWC2_COMPOSITION_SIDEBAND:对于这种 Layer,须要由内部机制提供内容更新,例如电视信号的视频数据。如果 HWC 不反对,则改为应用 CLIENT 或 DEVICE 形式合成,但可能无奈正确显示。

Cursor Layer还有一个重要的操作,setCursorPosition(),这个办法用来设置 Cursor 的地位,具体的实现仍然在 HWComposer 中。当用户过程更新 Surface 图形时,SurfaceFlinger 会发送INVALIDATE音讯给相应的 Layer。音讯解决函数调用handleTransaction()handlePageFlip()来更新Layer对象。handleTransaction()

用来解决 Layer 和显示设施的变动,它持续调用handleTransactionLocked()

frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

void SurfaceFlinger::handleTransactionLocked(uint32_t transactionFlags)
{
    ......
    // 解决Layer的变动
    if (transactionFlags & eTraversalNeeded) {
        ......
    }

    // 解决显示设施的变动
    if (transactionFlags & eDisplayTransactionNeeded) {
        processDisplayChangesLocked();
        processDisplayHotplugEventsLocked();
    }

    // 设置transform hint
    if (transactionFlags & (eDisplayLayerStackChanged|eDisplayTransactionNeeded)) {
        ......
   }

   //解决Layer的增减
   if (mLayersAdded) {
        ......
    }

    if (mLayersRemoved) {
        ......
    }

    commitTransaction();

    // 更新光标地位
    updateCursorAsync();
}

咱们找到了 Cursor 更新的中央,SurfaceFlinger 更新图形时会同步更新光标地位。之后,在 Vsync 到来时,实现图像的更新显示。

frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

void SurfaceFlinger::updateCursorAsync()
{
    for (size_t displayId = 0; displayId < mDisplays.size(); ++displayId) {
        ......
        // 调用Layer的对应办法
        for (auto& layer : displayDevice->getVisibleLayersSortedByZ()) {
            layer->updateCursorPosition(displayDevice);
        }
    }
}
frameworks/native/services/surfaceflinger/Layer.cpp

void Layer::updateCursorPosition(const sp<const DisplayDevice>& displayDevice) {
    // HWC Layer不存在或者不是Cursor Layer,不做解决
    auto hwcId = displayDevice->getHwcDisplayId();
    if (getBE().mHwcLayers.count(hwcId) == 0 ||
        getCompositionType(hwcId) != HWC2::Composition::Cursor) {
        return;
    }
    ......
    // 获取图层的地位
    Rect bounds = reduce(win, s.activeTransparentRegion);
    Rect frame(getTransform().transform(bounds));
    frame.intersect(displayDevice->getViewport(), &frame);
    if (!s.finalCrop.isEmpty()) {
        frame.intersect(s.finalCrop, &frame);
    }
    auto& displayTransform(displayDevice->getTransform());
    auto position = displayTransform.transform(frame);

    // 调用HWC的办法来设置图层地位
    auto error = getBE().mHwcLayers[hwcId].layer->setCursorPosition(position.left, position.top);
}

达到 HWComposer

下面剖析了许多代码,但真正与 Cursor 相干的并不多。CursorLayer 的真正实现还是在 HWComposer 中。然而 HWComposer 的实现是与平台相干的,不同的平台对 CursorLayer 的实现可能不同。效率的形式是应用一个独立的硬件 OSD 来显示 CursorLayer,而后通过硬件合成的形式将 CursorLayer 叠加到 UI 显示层。应用这种形式,光标的挪动效率也很高,只有扭转硬件 OSD 显示的地位即可。如果没有独立的硬件 OSD 来应用,就只能在规范显示层上进行软件叠加,或者应用 GPU 来叠加。

因为跟平台相干的实现具备私密性,这里不再持续剖析。

参考文档:

Android GUI零碎之SurfaceFlinger)

Android SurfaceFlinger学习-HWComposer Composition工作流程

挪动端显示技术杂谈

【腾讯云】轻量 2核2G4M,首年65元

阿里云限时活动-云数据库 RDS MySQL  1核2G配置 1.88/月 速抢

本文由乐趣区整理发布,转载请注明出处,谢谢。

您可能还喜欢...

发表回复

您的电子邮箱地址不会被公开。 必填项已用*标注

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据