乐趣区

关于jedis:聊聊jedis的borrow行为

本文次要钻研一下 jedis 的 borrow 行为

borrowObject

org/apache/commons/pool2/impl/GenericObjectPool.java

    public T borrowObject(final Duration borrowMaxWaitDuration) throws Exception {assertOpen();

        final AbandonedConfig ac = this.abandonedConfig;
        if (ac != null && ac.getRemoveAbandonedOnBorrow() && (getNumIdle() < 2) &&
                (getNumActive() > getMaxTotal() - 3)) {removeAbandoned(ac);
        }

        PooledObject<T> p = null;

        // Get local copy of current config so it is consistent for entire
        // method execution
        final boolean blockWhenExhausted = getBlockWhenExhausted();

        boolean create;
        final long waitTimeMillis = System.currentTimeMillis();

        while (p == null) {
            create = false;
            p = idleObjects.pollFirst();
            if (p == null) {p = create();
                if (p != null) {create = true;}
            }
            if (blockWhenExhausted) {if (p == null) {if (borrowMaxWaitDuration.isNegative()) {p = idleObjects.takeFirst();
                    } else {p = idleObjects.pollFirst(borrowMaxWaitDuration);
                    }
                }
                if (p == null) {
                    throw new NoSuchElementException(appendStats("Timeout waiting for idle object, borrowMaxWaitDuration=" + borrowMaxWaitDuration));
                }
            } else if (p == null) {throw new NoSuchElementException(appendStats("Pool exhausted"));
            }
            if (!p.allocate()) {p = null;}

            if (p != null) {
                try {factory.activateObject(p);
                } catch (final Exception e) {
                    try {destroy(p, DestroyMode.NORMAL);
                    } catch (final Exception e1) {// Ignore - activation failure is more important}
                    p = null;
                    if (create) {
                        final NoSuchElementException nsee = new NoSuchElementException(appendStats("Unable to activate object"));
                        nsee.initCause(e);
                        throw nsee;
                    }
                }
                if (p != null && getTestOnBorrow()) {
                    boolean validate = false;
                    Throwable validationThrowable = null;
                    try {validate = factory.validateObject(p);
                    } catch (final Throwable t) {PoolUtils.checkRethrow(t);
                        validationThrowable = t;
                    }
                    if (!validate) {
                        try {destroy(p, DestroyMode.NORMAL);
                            destroyedByBorrowValidationCount.incrementAndGet();} catch (final Exception e) {// Ignore - validation failure is more important}
                        p = null;
                        if (create) {
                            final NoSuchElementException nsee = new NoSuchElementException(appendStats("Unable to validate object"));
                            nsee.initCause(validationThrowable);
                            throw nsee;
                        }
                    }
                }
            }
        }

        updateStatsBorrow(p, Duration.ofMillis(System.currentTimeMillis() - waitTimeMillis));

        return p.getObject();}
  • borrowObject 办法会开启一个 while 循环,条件是 p 为 null,也就是要获取到 p 或者是外部本人跳出循环;idleObjects.pollFirst()从连接池获取,如果为 null 则执行 create,之后是 blockWhenExhausted 的判断逻辑,如果 create 进去的为 null,则阻塞期待 takeFirst 或者 pollFirst(borrowMaxWaitDuration),如果还是 null 则抛出 NoSuchElementException;如果 blockWhenExhausted 为 false 然而 create 为 null 则抛出Pool exhausted
  • 如果不是 null,则再次确认下 object 的状态,如果变更状态 (PooledObjectState.IDLE-->PooledObjectState.ALLOCATED) 不胜利则返回 null;接着执行 factory.activateObject(p)办法,如果出现异常则 destory 掉(jedis 这里只是在 db 不一样的时候会从新 select,默认能够了解为空操作),紧接着是 testOnBorrow 的逻辑
  • 这里就是如果 idleObjects.pollFirst()为 null 会触发 create,如果还是 null 则间接抛出 NoSuchElementException 异样,跳出循环;只有在不为 null 且 allocate 失败的时候会重置为 null 持续循环;另外如果是 create 进去的然而 activate 不胜利也会抛出 NoSuchElementException 异样,跳出循环

create

    /**
     * Attempts to create a new wrapped pooled object.
     * <p>
     * If there are {@link #getMaxTotal()} objects already in circulation
     * or in process of being created, this method returns null.
     * </p>
     *
     * @return The new wrapped pooled object
     *
     * @throws Exception if the object factory's {@code makeObject} fails
     */
    private PooledObject<T> create() throws Exception {int localMaxTotal = getMaxTotal();
        // This simplifies the code later in this method
        if (localMaxTotal < 0) {localMaxTotal = Integer.MAX_VALUE;}

        final long localStartTimeMillis = System.currentTimeMillis();
        final long localMaxWaitTimeMillis = Math.max(getMaxWaitDuration().toMillis(), 0);

        // Flag that indicates if create should:
        // - TRUE:  call the factory to create an object
        // - FALSE: return null
        // - null:  loop and re-test the condition that determines whether to
        //          call the factory
        Boolean create = null;
        while (create == null) {synchronized (makeObjectCountLock) {final long newCreateCount = createCount.incrementAndGet();
                if (newCreateCount > localMaxTotal) {
                    // The pool is currently at capacity or in the process of
                    // making enough new objects to take it to capacity.
                    createCount.decrementAndGet();
                    if (makeObjectCount == 0) {// There are no makeObject() calls in progress so the
                        // pool is at capacity. Do not attempt to create a new
                        // object. Return and wait for an object to be returned
                        create = Boolean.FALSE;
                    } else {// There are makeObject() calls in progress that might
                        // bring the pool to capacity. Those calls might also
                        // fail so wait until they complete and then re-test if
                        // the pool is at capacity or not.
                        makeObjectCountLock.wait(localMaxWaitTimeMillis);
                    }
                } else {
                    // The pool is not at capacity. Create a new object.
                    makeObjectCount++;
                    create = Boolean.TRUE;
                }
            }

            // Do not block more if maxWaitTimeMillis is set.
            if (create == null &&
                (localMaxWaitTimeMillis > 0 &&
                 System.currentTimeMillis() - localStartTimeMillis >= localMaxWaitTimeMillis)) {create = Boolean.FALSE;}
        }

        if (!create.booleanValue()) {return null;}

        final PooledObject<T> p;
        try {p = factory.makeObject();
            if (getTestOnCreate() && !factory.validateObject(p)) {createCount.decrementAndGet();
                return null;
            }
        } catch (final Throwable e) {createCount.decrementAndGet();
            throw e;
        } finally {synchronized (makeObjectCountLock) {
                makeObjectCount--;
                makeObjectCountLock.notifyAll();}
        }

        final AbandonedConfig ac = this.abandonedConfig;
        if (ac != null && ac.getLogAbandoned()) {p.setLogAbandoned(true);
            p.setRequireFullStackTrace(ac.getRequireFullStackTrace());
        }

        createdCount.incrementAndGet();
        allObjects.put(new IdentityWrapper<>(p.getObject()), p);
        return p;
    }

create 办法不会判断 createCount,如果超出则返回 null,如果期待超出 maxWait 也会返回 null;如果判断要创立则通过 factory.makeObject(),另外针对 testOnCreate 且 validateObject 不通过的也返回 null,如果是有异样则间接抛出

makeObject

redis/clients/jedis/JedisFactory.java

  @Override
  public PooledObject<Jedis> makeObject() throws Exception {
    Jedis jedis = null;
    try {jedis = new Jedis(jedisSocketFactory, clientConfig);
      jedis.connect();
      return new DefaultPooledObject<>(jedis);
    } catch (JedisException je) {if (jedis != null) {
        try {jedis.quit();
        } catch (RuntimeException e) {logger.warn("Error while QUIT", e);
        }
        try {jedis.close();
        } catch (RuntimeException e) {logger.warn("Error while close", e);
        }
      }
      throw je;
    }
  }

JedisFactory 的 makeObject 会创立 Jedis 而后执行 connect,如果有 JedisException 则抛出,这个也会间接跳出 borrowObject 的循环,间接给到调用方

activateObject

redis/clients/jedis/JedisFactory.java

  public void activateObject(PooledObject<Jedis> pooledJedis) throws Exception {final BinaryJedis jedis = pooledJedis.getObject();
    if (jedis.getDB() != clientConfig.getDatabase()) {jedis.select(clientConfig.getDatabase());
    }
  }

JedisFactory 的 activateObject 就是判断 db 跟配置的是不是一样,不一样则从新 select

testOnBorrow

                if (p != null && getTestOnBorrow()) {
                    boolean validate = false;
                    Throwable validationThrowable = null;
                    try {validate = factory.validateObject(p);
                    } catch (final Throwable t) {PoolUtils.checkRethrow(t);
                        validationThrowable = t;
                    }
                    if (!validate) {
                        try {destroy(p, DestroyMode.NORMAL);
                            destroyedByBorrowValidationCount.incrementAndGet();} catch (final Exception e) {// Ignore - validation failure is more important}
                        p = null;
                        if (create) {
                            final NoSuchElementException nsee = new NoSuchElementException(appendStats("Unable to validate object"));
                            nsee.initCause(validationThrowable);
                            throw nsee;
                        }
                    }
                }

    public static void checkRethrow(final Throwable t) {if (t instanceof ThreadDeath) {throw (ThreadDeath) t;
        }
        if (t instanceof VirtualMachineError) {throw (VirtualMachineError) t;
        }
        // All other instances of Throwable will be silently swallowed
    }

testOnBorrow 的逻辑就是执行 validateObject 办法,如果是 ThreadDeath 或者 VirtualMachineError 才会从新抛出,否则吞掉,之后判断 validate 后果,如果不胜利则执行 destory 办法,从新设置为 null,然而如果这个是 create 进去的则抛出 NoSuchElementException

小结

jedis 的 borrow 行为是在 while 循环外头去获取的,个别是在 allocate 变更状态不胜利 (PooledObjectState.IDLE-->PooledObjectState.ALLOCATED) 的时候会从新设置 null,持续循环

  • idleObjects.pollFirst()为 null 会触发 create,如果还是 null 则抛出 NoSuchElementException(Pool exhausted)跳出循环;如果 blockWhenExhausted 为 true,block 之后获取到的还是 null,也会抛出 NoSuchElementException(Timeout waiting for idle object)跳出循环;如果触发 create 操作,且 create 抛出 JedisException,这个也会间接跳出 borrowObject 的循环,间接给到调用方
  • borrow 进去不会 null 的执行 activateObject,jedis 这里只是在 db 不一样的时候会从新 select,默认能够了解为空操作
  • 最初是 testOnBorrow 的逻辑,如果有异样,则针对 create 进去的则抛出 NoSuchElementException 跳出循环,否则重置为 null 持续循环

    总结一下就是如果是 create 有异样 (JedisException) 则间接抛出,如果 borrow 不到 (即便通过 create) 也会抛出 NoSuchElementException(具体可能是 Pool exhausted 或者 Timeout waiting for idle object),如果有 testOnBorrow 不通过且是 create 进去的,也会抛出 NoSuchElementException(Unable to validate object)

退出移动版