关于jedis:聊聊jedis连接池参数配置

9次阅读

共计 10475 个字符，预计需要花费 27 分钟才能阅读完成。

本文次要钻研一下 jedis 连接池的参数配置

redis/clients/jedis/JedisPoolConfig.java

public class JedisPoolConfig extends GenericObjectPoolConfig<Jedis> {public JedisPoolConfig() {// defaults to make your life with connection pool easier :)
    setTestWhileIdle(true);
    setMinEvictableIdleTimeMillis(60000);
    setTimeBetweenEvictionRunsMillis(30000);
    setNumTestsPerEvictionRun(-1);
  }
}

JedisPoolConfig 继承了 GenericObjectPoolConfig，同时默认配置了 testWhileIdle 为 true(默认为 false)，minEvictableIdleTime 为 60s(默认为 30 分钟 )，timeBetweenEvictionRuns 为 30s( 默认为 -1)，numTestsPerEvictionRun 为 -1(即检测所有闲暇连贯，默认值为 3 )

org/apache/commons/pool2/impl/GenericObjectPoolConfig.java

public class GenericObjectPoolConfig<T> extends BaseObjectPoolConfig<T> {
    /**
     * The default value for the {@code maxTotal} configuration attribute.
     * @see GenericObjectPool#getMaxTotal()
     */
    public static final int DEFAULT_MAX_TOTAL = 8;

    /**
     * The default value for the {@code maxIdle} configuration attribute.
     * @see GenericObjectPool#getMaxIdle()
     */
    public static final int DEFAULT_MAX_IDLE = 8;

    /**
     * The default value for the {@code minIdle} configuration attribute.
     * @see GenericObjectPool#getMinIdle()
     */
    public static final int DEFAULT_MIN_IDLE = 0;


    private int maxTotal = DEFAULT_MAX_TOTAL;

    private int maxIdle = DEFAULT_MAX_IDLE;

    private int minIdle = DEFAULT_MIN_IDLE;
}

GenericObjectPoolConfig 继承了 BaseObjectPoolConfig，其 maxTotal 为 8，maxIdle 为 8，minIdle 为 0

org/apache/commons/pool2/impl/BaseObjectPoolConfig.java

public abstract class BaseObjectPoolConfig<T> extends BaseObject implements Cloneable {
    private boolean lifo = DEFAULT_LIFO;
    public static final boolean DEFAULT_LIFO = true;

    private boolean fairness = DEFAULT_FAIRNESS;
    public static final boolean DEFAULT_FAIRNESS = false;

    private Duration maxWaitDuration = DEFAULT_MAX_WAIT;
    public static final Duration DEFAULT_MAX_WAIT = Duration.ofMillis(-1L);

    private Duration minEvictableIdleDuration = DEFAULT_MIN_EVICTABLE_IDLE_TIME;
    public static final Duration DEFAULT_MIN_EVICTABLE_IDLE_TIME =
            Duration.ofMillis(1000L * 60L * 30L);


    private Duration evictorShutdownTimeoutDuration = DEFAULT_EVICTOR_SHUTDOWN_TIMEOUT;
    public static final Duration DEFAULT_EVICTOR_SHUTDOWN_TIMEOUT =
            Duration.ofMillis(10L * 1000L);

    private Duration softMinEvictableIdleDuration = DEFAULT_SOFT_MIN_EVICTABLE_IDLE_TIME;
    public static final Duration DEFAULT_SOFT_MIN_EVICTABLE_IDLE_TIME =
            Duration.ofMillis(-1);

    private int numTestsPerEvictionRun = DEFAULT_NUM_TESTS_PER_EVICTION_RUN;
    public static final int DEFAULT_NUM_TESTS_PER_EVICTION_RUN = 3;

    private EvictionPolicy<T> evictionPolicy; // Only 2.6.0 applications set this

    private String evictionPolicyClassName = DEFAULT_EVICTION_POLICY_CLASS_NAME;
    public static final String DEFAULT_EVICTION_POLICY_CLASS_NAME = DefaultEvictionPolicy.class.getName();

    private boolean testOnCreate = DEFAULT_TEST_ON_CREATE;
    public static final boolean DEFAULT_TEST_ON_CREATE = false;

    private boolean testOnBorrow = DEFAULT_TEST_ON_BORROW;
    public static final boolean DEFAULT_TEST_ON_BORROW = false;

    private boolean testOnReturn = DEFAULT_TEST_ON_RETURN;
    public static final boolean DEFAULT_TEST_ON_RETURN = false;

    private boolean testWhileIdle = DEFAULT_TEST_WHILE_IDLE;
    public static final boolean DEFAULT_TEST_WHILE_IDLE = false;

    private Duration durationBetweenEvictionRuns = DEFAULT_TIME_BETWEEN_EVICTION_RUNS;
    public static final Duration DEFAULT_TIME_BETWEEN_EVICTION_RUNS = Duration
            .ofMillis(-1L);

    private boolean blockWhenExhausted = DEFAULT_BLOCK_WHEN_EXHAUSTED;
    public static final boolean DEFAULT_BLOCK_WHEN_EXHAUSTED = true;

    private boolean jmxEnabled = DEFAULT_JMX_ENABLE;
    public static final boolean DEFAULT_JMX_ENABLE = true;

    // TODO Consider changing this to a single property for 3.x
    private String jmxNamePrefix = DEFAULT_JMX_NAME_PREFIX;
    public static final String DEFAULT_JMX_NAME_PREFIX = "pool";

    private String jmxNameBase = DEFAULT_JMX_NAME_BASE;
    public static final String DEFAULT_JMX_NAME_BASE = null;
}

BaseObjectPoolConfig 定义了 lifo(默认为 true)、fairness(默认为 false)、maxWaitDuration(默认为 -1)、minEvictableIdleDuration(默认为 30 分钟 )、evictorShutdownTimeoutDuration( 默认为 10s)、softMinEvictableIdleDuration(默认为 -1)、numTestsPerEvictionRun(默认为 3 )、evictionPolicyClassName(默认为 DefaultEvictionPolicy.class.getName())、testOnCreate(默认为 false)、testOnBorrow(默认为 false)、testOnReturn(默认为 false)、testWhileIdle(默认为 false)、durationBetweenEvictionRuns(默认为 -1)、blockWhenExhausted(默认为 true)、jmxEnabled(默认为 true)、jmxNamePrefix(默认为 pool)、jmxNameBase(默认为 null)

org/apache/commons/pool2/impl/GenericObjectPool.java

            if (blockWhenExhausted) {if (p == null) {if (borrowMaxWaitDuration.isNegative()) {p = idleObjects.takeFirst();
                    } else {p = idleObjects.pollFirst(borrowMaxWaitDuration);
                    }
                }
                if (p == null) {
                    throw new NoSuchElementException(appendStats("Timeout waiting for idle object, borrowMaxWaitDuration=" + borrowMaxWaitDuration));
                }
            }

borrow 的时候在 blockWhenExhausted 为 true 时应用，- 1 的话则应用 takeFirst 办法，否则应用 pollFirst(borrowMaxWaitDuration)办法

org/apache/commons/pool2/impl/GenericObjectPool.java

    /**
     * Calculates the number of objects to test in a run of the idle object
     * evictor.
     *
     * @return The number of objects to test for validity
     */
    private int getNumTests() {final int numTestsPerEvictionRun = getNumTestsPerEvictionRun();
        if (numTestsPerEvictionRun >= 0) {return Math.min(numTestsPerEvictionRun, idleObjects.size());
        }
        return (int) (Math.ceil(idleObjects.size() /
                Math.abs((double) numTestsPerEvictionRun)));
    }

当 numTestsPerEvictionRun 为 - 1 时，返回的是 idleObjects.size()

org/apache/commons/pool2/impl/EvictionConfig.java

public class EvictionConfig {private static final Duration MAX_DURATION = Duration.ofMillis(Long.MAX_VALUE);
    private final Duration idleEvictDuration;
    private final Duration idleSoftEvictDuration;
    private final int minIdle;

    /**
     * Creates a new eviction configuration with the specified parameters.
     * Instances are immutable.
     *
     * @param idleEvictDuration Expected to be provided by
     *        {@link BaseGenericObjectPool#getMinEvictableIdleDuration()}
     * @param idleSoftEvictDuration Expected to be provided by
     *        {@link BaseGenericObjectPool#getSoftMinEvictableIdleDuration()}
     * @param minIdle Expected to be provided by
     *        {@link GenericObjectPool#getMinIdle()} or
     *        {@link GenericKeyedObjectPool#getMinIdlePerKey()}
     * @since 2.10.0
     */
    public EvictionConfig(final Duration idleEvictDuration, final Duration idleSoftEvictDuration, final int minIdle) {this.idleEvictDuration = PoolImplUtils.isPositive(idleEvictDuration) ? idleEvictDuration : MAX_DURATION;
        this.idleSoftEvictDuration = PoolImplUtils.isPositive(idleSoftEvictDuration) ? idleSoftEvictDuration : MAX_DURATION;
        this.minIdle = minIdle;
    }

    //......
}

evict 办法会结构 EvictionConfig，minEvictableIdleDuration 默认为 30 分钟；这里次要是针对正数进行了判断，正数则取的 Long.MAX_VALUE；即 softMinEvictableIdleDuration 默认为 -1，转换过去就是 Long.MAX_VALUE

final EvictionConfig evictionConfig = new EvictionConfig(getMinEvictableIdleDuration(),
                        getSoftMinEvictableIdleDuration(),
                        getMinIdle());

org/apache/commons/pool2/impl/DefaultEvictionPolicy.java

public class DefaultEvictionPolicy<T> implements EvictionPolicy<T> {

    @Override
    public boolean evict(final EvictionConfig config, final PooledObject<T> underTest, final int idleCount) {
        // @formatter:off
        return (config.getIdleSoftEvictDuration().compareTo(underTest.getIdleDuration()) < 0 &&
                config.getMinIdle() < idleCount) ||
                config.getIdleEvictDuration().compareTo(underTest.getIdleDuration()) < 0;
        // @formatter:on
    }
}

DefaultEvictionPolicy 的 evict 办法在 idleCount 大于 minIdle 且 idleDuration 大于 idleSoftEvictDuration(默认为 Long.MAX_VALUE)返回 true，或者 obj 的 idleDuration 大于 minEvictableIdleDuration(默认为 30 分钟 ) 返回 true

org/apache/commons/pool2/impl/BaseGenericObjectPool.java

    public final void setTimeBetweenEvictionRuns(final Duration timeBetweenEvictionRuns) {this.durationBetweenEvictionRuns = PoolImplUtils.nonNull(timeBetweenEvictionRuns, BaseObjectPoolConfig.DEFAULT_TIME_BETWEEN_EVICTION_RUNS);
        startEvictor(this.durationBetweenEvictionRuns);
    }

    final void startEvictor(final Duration delay) {synchronized (evictionLock) {final boolean isPositiverDelay = PoolImplUtils.isPositive(delay);
            if (evictor == null) { // Starting evictor for the first time or after a cancel
                if (isPositiverDelay) { // Starting new evictor
                    evictor = new Evictor();
                    EvictionTimer.schedule(evictor, delay, delay);
                }
            } else if (isPositiverDelay) { // Stop or restart of existing evictor: Restart
                synchronized (EvictionTimer.class) { // Ensure no cancel can happen between cancel / schedule calls
                    EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, true);
                    evictor = null;
                    evictionIterator = null;
                    evictor = new Evictor();
                    EvictionTimer.schedule(evictor, delay, delay);
                }
            } else { // Stopping evictor
                EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, false);
            }
        }
    }

setTimeBetweenEvictionRuns 办法会 durationBetweenEvictionRuns(默认为 -1)去 startEvictor；当改值为正数时则执行 EvictionTimer.cancel，即不启动 evictor 线程

org/apache/commons/pool2/impl/EvictionTimer.java

    /**
     * Removes the specified eviction task from the timer.
     *
     * @param evictor   Task to be cancelled.
     * @param timeout   If the associated executor is no longer required, how
     *                  long should this thread wait for the executor to
     *                  terminate?
     * @param restarting The state of the evictor.
     */
    static synchronized void cancel(final BaseGenericObjectPool<?>.Evictor evictor, final Duration timeout,
            final boolean restarting) {if (evictor != null) {evictor.cancel();
            remove(evictor);
        }
        if (!restarting && executor != null && taskMap.isEmpty()) {executor.shutdown();
            try {executor.awaitTermination(timeout.toMillis(), TimeUnit.MILLISECONDS);
            } catch (final InterruptedException e) {
                // Swallow
                // Significant API changes would be required to propagate this
            }
            executor.setCorePoolSize(0);
            executor = null;
        }
    }

evictorShutdownTimeoutDuration 即 executor.awaitTermination 的 timeout 参数

参数类别	参数值	jedisConfig 值	commons-pool 默认值	解释
根本	lifo	true	true	后进先出
根本	fairness	false	false	非偏心机制
数量	maxTotal	8	–	最大连接数
数量	maxIdle	8	–	最大空间连接数
数量	minIdle	0	–	起码闲暇连接数
阻塞获取	blockWhenExhausted	true	true	连接池耗尽时，获取连贯是否阻塞期待
阻塞获取	maxWaitMillis	-1	-1	blockWhenExhausted 为 true 时阻塞期待多久
衰弱检测	testOnCreate	fasle	fasle	创立连贯的时候检测
衰弱检测	testOnBorrow	false	false	借用连贯的时候检测
衰弱检测	testOnReturn	false	false	偿还连贯的时候检测
衰弱检测	testWhileIdle	true	false	在闲暇连贯检测时检测
evict	durationBetweenEvictionRuns	30 秒	-1	evictor 线程运行距离，- 1 为不运行
evict	minEvictableIdleDuration	60 秒	30 分钟	连贯闲暇的最小工夫
evict	softMinEvictableIdleDuration	-1	-1	- 1 即 Long.MAX_VALUE，它与 idleCount>minIdle 条件一起思考
evict	numTestsPerEvictionRun	-1	3	每次闲暇连贯回收器线程 (如果有) 运行时查看的连贯数量, - 1 示意全副
evict	evictionPolicyClassName	DefaultEvictionPolicy	DefaultEvictionPolicy	判断是否须要 evict 的 PolicyClass
evict	evictorShutdownTimeoutDuration	10s	10s	敞开 evictor 线程池的等待时间
jmx	jmxEnabled	true	true	是否开启 jmx
jmx	jmxNamePrefix	pool	pool	jmx 名称前缀
jmx	jmxNameBase	null	null	null 示意由 pool 本人定义 jmxNameBase

JedisConfig 默认帮咱们配置了 testWhileIdle 为 true(默认为 false)，minEvictableIdleTime 为 60s(默认为 30 分钟 )，timeBetweenEvictionRuns 为 30s( 默认为 -1)，numTestsPerEvictionRun 为 -1(即检测所有闲暇连贯，默认值为 3 )，利用 evictor 线程来检测闲暇连贯的衰弱状况

另外由聊聊 jedis 的 return 行为这篇剖析能够得悉在执行命令时若 redis 出问题，Jedis 自身会标记底层 connection 为 broken，在 finally 偿还时会 destory 连贯，保障连接池连贯最终都会被清空重建。

另外在并发量比拟大的场景，若要保障连接池的稳固数量则能够把 minIdle 设置成与 maxTotal 和 maxIdle 一样即可；若不想因为 idle 工夫被频繁 destory 则能够设置 minEvictableIdleTime 为 -1，evict 办法始终返回 false，evictor 线程始终走的是 testWhileIdle 的逻辑；maxWaitMillis 要设置一个正当值防止连接池耗尽阻塞线程，或者间接设置 blockWhenExhausted 为 false

GenericObjectPool 参数解析
JedisPool 资源池优化
一次拜访 Redis 延时高问题排查与总结
实战总结|一次拜访 Redis 延时高问题排查与总结（续）

正文完

jedis

发表至： jedis

2023-09-24

0

关于jedis:聊聊jedis的return行为

关于jedis:聊聊jedis的borrow行为

关于jedis:聊聊jedis连接池参数配置

关于jedis:聊聊jedis的testWhileIdle

关于程序员:算法基础之高精度总结

关于jedis:聊聊jedis连接池参数配置

序

JedisConfig

GenericObjectPoolConfig

BaseObjectPoolConfig

maxWaitDuration

evict

getNumTests

EvictionConfig

DefaultEvictionPolicy

durationBetweenEvictionRuns

cancel

小结

doc

站内搜索