更多技术文章，请关注我的集体博客 www.immaxfang.com 和小公众号 Max的学习札记。

概述

Redis 的 client-output-buffer-limit 能够用来强制断开无奈足够快从 redis 服务器端读取数据的客户端。
爱护机制规定如下：

  1. [hard limit] 大小限度，当某一客户端缓冲区超过设定值后，间接敞开连贯。
  2. [soft limit] 持续时间限度，当某一客户端缓冲区继续一段时间占用过大空间时敞开连贯。

该参数个别用在以下几类客户端中：

  • 一般 client，包含 monitor
  • 主从同步时的 slave client

  • Pub/Sub 模式中的 client

    配置介绍与剖析

    该参数的配置语法：

    client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>

    配置实例：

    # 一般client buffer限度 client-output-buffer-limit normal 0 0 0# slave client buffer限度client-output-buffer-limit slave 256mb 64mb 60# pubsub client buffer限度client-output-buffer-limit pubsub 32mb 8mb 60
  • client-output-buffer-limit normal 0 0 0

将 hard limit 和 soft limit 同时设置为 0，则示意敞开该限度。

  • client-output-buffer-limit slave 256mb 64mb 60

该配置示意，对于 slave 客户端来说，如果 output-buffer 占用内存达到 256M 或者超过 64M 的工夫达到 60s，则敞开客户端连贯。

  • client-output-buffer-limit pubsub 32mb 8mb 60

该配置示意，对于 Pub/Sub 客户端来说，若 output-buffer 占用内存达到 32M 或者超过 8M 的工夫达到 60s，则敞开客户端连贯。

个别状况下，对于一般客户端，client-output-buffer 是不设限度的，因为 server 只会在 client 申请数据的时候才会发送，不会产生积压。
而在 server 被动发送，client 来解决的场景下，这种个别都是异步解决的，会划出一个缓冲区来“暂存”未解决的数据，若 server 发送数据比 client 解决数据快时，就会产生缓冲区积压。对于用作 Pub/Sub 和 slave 的客户端，server 会被动把数据推送给他们，故须要设置 client-output-buffer 的限度。

示例剖析

上面咱们以主从同步时的 slave 客户端，来具体分析下。
在 redis 在主从同步时，master 会为 slave 创立一个输入缓冲区。在 master 保留 rdb，将 rdb 文件传输给 slave，slave 加载 rdb 实现之前，master 会将接管到的所有写命令，写入到内存中的这个输入缓冲区去。
若 rdb 的保留，传输，加载耗时过长，或者在此期间的写命令过多，则可能会造成超过缓冲区限度，造成 master 和 slave 的连贯断开。此时则须要适当调整下 client-output-buffer-limit slave配置。

源码浅析-主从同步时 output buffer 应用

基于 redis5.0 版本源码

redis server 通过 addReply 将数据发送给客户端，以下源码见 https://github.com/redis/redis/blob/5.0/src/networking.c#L190-L211

/* Add the object 'obj' string representation to the client output buffer. */void addReply(client *c, robj *obj) {    if (prepareClientToWrite(c) != C_OK) return;    if (sdsEncodedObject(obj)) {        if (_addReplyToBuffer(c,obj->ptr,sdslen(obj->ptr)) != C_OK)            _addReplyStringToList(c,obj->ptr,sdslen(obj->ptr));    } else if (obj->encoding == OBJ_ENCODING_INT) {        char buf[32];        size_t len = ll2string(buf,sizeof(buf),(long)obj->ptr);        if (_addReplyToBuffer(c,buf,len) != C_OK)            _addReplyStringToList(c,buf,len);    } else {        serverPanic("Wrong obj->encoding in addReply()");    }}

在函数的结尾，会通过prepareClientToWrite(c)判断是否须要将数据写入客户端的 output buffer 中。咱们看下什么条件下数据会被写入客户端的 output buffer 中，即返回 C_OK。

/* This function is called every time we are going to transmit new data * to the client. The behavior is the following: * * If the client should receive new data (normal clients will) the function * returns C_OK, and make sure to install the write handler in our event * loop so that when the socket is writable new data gets written. * * If the client should not receive new data, because it is a fake client * (used to load AOF in memory), a master or because the setup of the write * handler failed, the function returns C_ERR. * * The function may return C_OK without actually installing the write * event handler in the following cases: * * 1) The event handler should already be installed since the output buffer *    already contains something. * 2) The client is a slave but not yet online, so we want to just accumulate *    writes in the buffer but not actually sending them yet. * * Typically gets called every time a reply is built, before adding more * data to the clients output buffers. If the function returns C_ERR no * data should be appended to the output buffers. */int prepareClientToWrite(client *c) {    /* If it's the Lua client we always return ok without installing any     * handler since there is no socket at all. */    if (c->flags & (CLIENT_LUA|CLIENT_MODULE)) return C_OK;    /* CLIENT REPLY OFF / SKIP handling: don't send replies. */    if (c->flags & (CLIENT_REPLY_OFF|CLIENT_REPLY_SKIP)) return C_ERR;    /* Masters don't receive replies, unless CLIENT_MASTER_FORCE_REPLY flag     * is set. */    if ((c->flags & CLIENT_MASTER) &&        !(c->flags & CLIENT_MASTER_FORCE_REPLY)) return C_ERR;    if (c->fd <= 0) return C_ERR; /* Fake client for AOF loading. */    /* Schedule the client to write the output buffers to the socket, unless     * it should already be setup to do so (it has already pending data). */    if (!clientHasPendingReplies(c)) clientInstallWriteHandler(c);    /* Authorize the caller to queue in the output buffer of this client. */    return C_OK;}/* Return true if the specified client has pending reply buffers to write to * the socket. */int clientHasPendingReplies(client *c) {    return c->bufpos || listLength(c->reply);}void clientInstallWriteHandler(client *c) {    /* Schedule the client to write the output buffers to the socket only     * if not already done and, for slaves, if the slave can actually receive     * writes at this stage. */    if (!(c->flags & CLIENT_PENDING_WRITE) &&        (c->replstate == REPL_STATE_NONE ||         (c->replstate == SLAVE_STATE_ONLINE && !c->repl_put_online_on_ack)))    {        c->flags |= CLIENT_PENDING_WRITE;        listAddNodeHead(server.clients_pending_write,c);    }}

因为函数默认返回C_OK，咱们只须要看哪几类状况返回的不是C_OK，即C_ERR，数据就不会被写入到客户端的 output buffer 中。
返回C_ERR的状况：

  • 客户端是个 fake client（用于加载 AOF 文件）
  • 客户端是一个 master

  • slave 的状态为 SLAVE_STATE_ONLINE 且其回调函数失败（(c->replstate == SLAVE_STATE_ONLINE && !c->repl_put_online_on_ack)），或slave 的状态为 REPL_STATE_NONE

    If the client should not receive new data, because it is a fake client (used to load AOF in memory), a master or because the setup of the write handler failed, the function returns C_ERR.

在 master 保留和发送 rdb 文件时，slave 的状态是以下几种，所以在这期间的写命令都会保留在 slave 的 output buffer。因为没有设置回调函数，数据并不会发送到 slave 上，仅存储在 master 为 slave 创立的 output buffer 内。

#define SLAVE_STATE_WAIT_BGSAVE_START 6 /* We need to produce a new RDB file. */#define SLAVE_STATE_WAIT_BGSAVE_END 7 /* Waiting RDB file creation to finish. */#define SLAVE_STATE_SEND_BULK 8 /* Sending RDB file to slave. */

那么何时才会从 output buffer 中“刷入”slave 呢？直到 master 将 rdb 文件齐全发送给 slave 后，master 会在 sendBulkToSlave函数中进行相干操作。以下源码见：https://github.com/redis/redis/blob/5.0/src/replication.c#L876-L930

void sendBulkToSlave(aeEventLoop *el, int fd, void *privdata, int mask) {    // 此处省略局部源码    // rdb 文件已齐全发送给 slave     if (slave->repldboff == slave->repldbsize) {        close(slave->repldbfd);        slave->repldbfd = -1;        aeDeleteFileEvent(server.el,slave->fd,AE_WRITABLE);        putSlaveOnline(slave);    }}void putSlaveOnline(client *slave) {    slave->replstate = SLAVE_STATE_ONLINE;    slave->repl_put_online_on_ack = 0;    slave->repl_ack_time = server.unixtime; /* Prevent false timeout. */    if (aeCreateFileEvent(server.el, slave->fd, AE_WRITABLE,        sendReplyToClient, slave) == AE_ERR) {        serverLog(LL_WARNING,"Unable to register writable event for replica bulk transfer: %s", strerror(errno));        freeClient(slave);        return;    }    refreshGoodSlavesCount();    serverLog(LL_NOTICE,"Synchronization with replica %s succeeded",        replicationGetSlaveName(slave));}

此处会将 slave 状态改为 SLAVE_STATE_ONLINE，并将repl_put_online_on_ack置为0，（有没有很相熟，对了，就是下面clientInstallWriteHandler中判断的内容）。同时也会设置回调函数sendReplyToClient，将此前 master 为 slave 创立的 output buffer 中的写操作全副发送到 slave 上。同时 slave 状态的变更，会使得后续 master 上的写操作能够失常的 push 到 slave 上了（间接，无需走 output buffer）。

总结

本次咱们通过 client-output-buffer-limit参数，理解了其应用场景，并重点就主从同步时 output buffer 写入状况进行了源码的简略剖析。明天的学习就到这里，咱们改天接着肝。

参考内容

  1. https://www.cnblogs.com/wangcp-2014/p/15505180.html