共计 10475 个字符,预计需要花费 27 分钟才能阅读完成。
序
本文次要钻研一下 jedis 连接池的参数配置
JedisConfig
redis/clients/jedis/JedisPoolConfig.java
public class JedisPoolConfig extends GenericObjectPoolConfig<Jedis> {public JedisPoolConfig() {// defaults to make your life with connection pool easier :)
setTestWhileIdle(true);
setMinEvictableIdleTimeMillis(60000);
setTimeBetweenEvictionRunsMillis(30000);
setNumTestsPerEvictionRun(-1);
}
}
JedisPoolConfig 继承了 GenericObjectPoolConfig,同时默认配置了 testWhileIdle 为 true(
默认为 false
),minEvictableIdleTime 为 60s(默认为 30 分钟
),timeBetweenEvictionRuns 为 30s(默认为 -1
),numTestsPerEvictionRun 为 -1(即检测所有闲暇连贯,默认值为 3
)
GenericObjectPoolConfig
org/apache/commons/pool2/impl/GenericObjectPoolConfig.java
public class GenericObjectPoolConfig<T> extends BaseObjectPoolConfig<T> {
/**
* The default value for the {@code maxTotal} configuration attribute.
* @see GenericObjectPool#getMaxTotal()
*/
public static final int DEFAULT_MAX_TOTAL = 8;
/**
* The default value for the {@code maxIdle} configuration attribute.
* @see GenericObjectPool#getMaxIdle()
*/
public static final int DEFAULT_MAX_IDLE = 8;
/**
* The default value for the {@code minIdle} configuration attribute.
* @see GenericObjectPool#getMinIdle()
*/
public static final int DEFAULT_MIN_IDLE = 0;
private int maxTotal = DEFAULT_MAX_TOTAL;
private int maxIdle = DEFAULT_MAX_IDLE;
private int minIdle = DEFAULT_MIN_IDLE;
}
GenericObjectPoolConfig 继承了 BaseObjectPoolConfig,其 maxTotal 为 8,maxIdle 为 8,minIdle 为 0
BaseObjectPoolConfig
org/apache/commons/pool2/impl/BaseObjectPoolConfig.java
public abstract class BaseObjectPoolConfig<T> extends BaseObject implements Cloneable {
private boolean lifo = DEFAULT_LIFO;
public static final boolean DEFAULT_LIFO = true;
private boolean fairness = DEFAULT_FAIRNESS;
public static final boolean DEFAULT_FAIRNESS = false;
private Duration maxWaitDuration = DEFAULT_MAX_WAIT;
public static final Duration DEFAULT_MAX_WAIT = Duration.ofMillis(-1L);
private Duration minEvictableIdleDuration = DEFAULT_MIN_EVICTABLE_IDLE_TIME;
public static final Duration DEFAULT_MIN_EVICTABLE_IDLE_TIME =
Duration.ofMillis(1000L * 60L * 30L);
private Duration evictorShutdownTimeoutDuration = DEFAULT_EVICTOR_SHUTDOWN_TIMEOUT;
public static final Duration DEFAULT_EVICTOR_SHUTDOWN_TIMEOUT =
Duration.ofMillis(10L * 1000L);
private Duration softMinEvictableIdleDuration = DEFAULT_SOFT_MIN_EVICTABLE_IDLE_TIME;
public static final Duration DEFAULT_SOFT_MIN_EVICTABLE_IDLE_TIME =
Duration.ofMillis(-1);
private int numTestsPerEvictionRun = DEFAULT_NUM_TESTS_PER_EVICTION_RUN;
public static final int DEFAULT_NUM_TESTS_PER_EVICTION_RUN = 3;
private EvictionPolicy<T> evictionPolicy; // Only 2.6.0 applications set this
private String evictionPolicyClassName = DEFAULT_EVICTION_POLICY_CLASS_NAME;
public static final String DEFAULT_EVICTION_POLICY_CLASS_NAME = DefaultEvictionPolicy.class.getName();
private boolean testOnCreate = DEFAULT_TEST_ON_CREATE;
public static final boolean DEFAULT_TEST_ON_CREATE = false;
private boolean testOnBorrow = DEFAULT_TEST_ON_BORROW;
public static final boolean DEFAULT_TEST_ON_BORROW = false;
private boolean testOnReturn = DEFAULT_TEST_ON_RETURN;
public static final boolean DEFAULT_TEST_ON_RETURN = false;
private boolean testWhileIdle = DEFAULT_TEST_WHILE_IDLE;
public static final boolean DEFAULT_TEST_WHILE_IDLE = false;
private Duration durationBetweenEvictionRuns = DEFAULT_TIME_BETWEEN_EVICTION_RUNS;
public static final Duration DEFAULT_TIME_BETWEEN_EVICTION_RUNS = Duration
.ofMillis(-1L);
private boolean blockWhenExhausted = DEFAULT_BLOCK_WHEN_EXHAUSTED;
public static final boolean DEFAULT_BLOCK_WHEN_EXHAUSTED = true;
private boolean jmxEnabled = DEFAULT_JMX_ENABLE;
public static final boolean DEFAULT_JMX_ENABLE = true;
// TODO Consider changing this to a single property for 3.x
private String jmxNamePrefix = DEFAULT_JMX_NAME_PREFIX;
public static final String DEFAULT_JMX_NAME_PREFIX = "pool";
private String jmxNameBase = DEFAULT_JMX_NAME_BASE;
public static final String DEFAULT_JMX_NAME_BASE = null;
}
BaseObjectPoolConfig 定义了 lifo(
默认为 true
)、fairness(默认为 false
)、maxWaitDuration(默认为 -1
)、minEvictableIdleDuration(默认为 30 分钟
)、evictorShutdownTimeoutDuration(默认为 10s
)、softMinEvictableIdleDuration(默认为 -1
)、numTestsPerEvictionRun(默认为 3
)、evictionPolicyClassName(默认为 DefaultEvictionPolicy.class.getName()
)、testOnCreate(默认为 false
)、testOnBorrow(默认为 false
)、testOnReturn(默认为 false
)、testWhileIdle(默认为 false
)、durationBetweenEvictionRuns(默认为 -1
)、blockWhenExhausted(默认为 true
)、jmxEnabled(默认为 true
)、jmxNamePrefix(默认为 pool
)、jmxNameBase(默认为 null
)
maxWaitDuration
org/apache/commons/pool2/impl/GenericObjectPool.java
if (blockWhenExhausted) {if (p == null) {if (borrowMaxWaitDuration.isNegative()) {p = idleObjects.takeFirst();
} else {p = idleObjects.pollFirst(borrowMaxWaitDuration);
}
}
if (p == null) {
throw new NoSuchElementException(appendStats("Timeout waiting for idle object, borrowMaxWaitDuration=" + borrowMaxWaitDuration));
}
}
borrow 的时候在 blockWhenExhausted 为 true 时应用,- 1 的话则应用 takeFirst 办法,否则应用 pollFirst(borrowMaxWaitDuration)办法
evict
getNumTests
org/apache/commons/pool2/impl/GenericObjectPool.java
/**
* Calculates the number of objects to test in a run of the idle object
* evictor.
*
* @return The number of objects to test for validity
*/
private int getNumTests() {final int numTestsPerEvictionRun = getNumTestsPerEvictionRun();
if (numTestsPerEvictionRun >= 0) {return Math.min(numTestsPerEvictionRun, idleObjects.size());
}
return (int) (Math.ceil(idleObjects.size() /
Math.abs((double) numTestsPerEvictionRun)));
}
当 numTestsPerEvictionRun 为 - 1 时,返回的是 idleObjects.size()
EvictionConfig
org/apache/commons/pool2/impl/EvictionConfig.java
public class EvictionConfig {private static final Duration MAX_DURATION = Duration.ofMillis(Long.MAX_VALUE);
private final Duration idleEvictDuration;
private final Duration idleSoftEvictDuration;
private final int minIdle;
/**
* Creates a new eviction configuration with the specified parameters.
* Instances are immutable.
*
* @param idleEvictDuration Expected to be provided by
* {@link BaseGenericObjectPool#getMinEvictableIdleDuration()}
* @param idleSoftEvictDuration Expected to be provided by
* {@link BaseGenericObjectPool#getSoftMinEvictableIdleDuration()}
* @param minIdle Expected to be provided by
* {@link GenericObjectPool#getMinIdle()} or
* {@link GenericKeyedObjectPool#getMinIdlePerKey()}
* @since 2.10.0
*/
public EvictionConfig(final Duration idleEvictDuration, final Duration idleSoftEvictDuration, final int minIdle) {this.idleEvictDuration = PoolImplUtils.isPositive(idleEvictDuration) ? idleEvictDuration : MAX_DURATION;
this.idleSoftEvictDuration = PoolImplUtils.isPositive(idleSoftEvictDuration) ? idleSoftEvictDuration : MAX_DURATION;
this.minIdle = minIdle;
}
//......
}
evict 办法会结构 EvictionConfig,minEvictableIdleDuration 默认为 30 分钟;这里次要是针对正数进行了判断,正数则取的 Long.MAX_VALUE;即 softMinEvictableIdleDuration 默认为 -1,转换过去就是 Long.MAX_VALUE
final EvictionConfig evictionConfig = new EvictionConfig(getMinEvictableIdleDuration(),
getSoftMinEvictableIdleDuration(),
getMinIdle());
DefaultEvictionPolicy
org/apache/commons/pool2/impl/DefaultEvictionPolicy.java
public class DefaultEvictionPolicy<T> implements EvictionPolicy<T> {
@Override
public boolean evict(final EvictionConfig config, final PooledObject<T> underTest, final int idleCount) {
// @formatter:off
return (config.getIdleSoftEvictDuration().compareTo(underTest.getIdleDuration()) < 0 &&
config.getMinIdle() < idleCount) ||
config.getIdleEvictDuration().compareTo(underTest.getIdleDuration()) < 0;
// @formatter:on
}
}
DefaultEvictionPolicy 的 evict 办法在 idleCount 大于 minIdle 且 idleDuration 大于 idleSoftEvictDuration(
默认为 Long.MAX_VALUE
)返回 true,或者 obj 的 idleDuration 大于 minEvictableIdleDuration(默认为 30 分钟
) 返回 true
durationBetweenEvictionRuns
org/apache/commons/pool2/impl/BaseGenericObjectPool.java
public final void setTimeBetweenEvictionRuns(final Duration timeBetweenEvictionRuns) {this.durationBetweenEvictionRuns = PoolImplUtils.nonNull(timeBetweenEvictionRuns, BaseObjectPoolConfig.DEFAULT_TIME_BETWEEN_EVICTION_RUNS);
startEvictor(this.durationBetweenEvictionRuns);
}
final void startEvictor(final Duration delay) {synchronized (evictionLock) {final boolean isPositiverDelay = PoolImplUtils.isPositive(delay);
if (evictor == null) { // Starting evictor for the first time or after a cancel
if (isPositiverDelay) { // Starting new evictor
evictor = new Evictor();
EvictionTimer.schedule(evictor, delay, delay);
}
} else if (isPositiverDelay) { // Stop or restart of existing evictor: Restart
synchronized (EvictionTimer.class) { // Ensure no cancel can happen between cancel / schedule calls
EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, true);
evictor = null;
evictionIterator = null;
evictor = new Evictor();
EvictionTimer.schedule(evictor, delay, delay);
}
} else { // Stopping evictor
EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, false);
}
}
}
setTimeBetweenEvictionRuns 办法会 durationBetweenEvictionRuns(
默认为 -1
)去 startEvictor;当改值为正数时则执行 EvictionTimer.cancel,即不启动 evictor 线程
cancel
org/apache/commons/pool2/impl/EvictionTimer.java
/**
* Removes the specified eviction task from the timer.
*
* @param evictor Task to be cancelled.
* @param timeout If the associated executor is no longer required, how
* long should this thread wait for the executor to
* terminate?
* @param restarting The state of the evictor.
*/
static synchronized void cancel(final BaseGenericObjectPool<?>.Evictor evictor, final Duration timeout,
final boolean restarting) {if (evictor != null) {evictor.cancel();
remove(evictor);
}
if (!restarting && executor != null && taskMap.isEmpty()) {executor.shutdown();
try {executor.awaitTermination(timeout.toMillis(), TimeUnit.MILLISECONDS);
} catch (final InterruptedException e) {
// Swallow
// Significant API changes would be required to propagate this
}
executor.setCorePoolSize(0);
executor = null;
}
}
evictorShutdownTimeoutDuration 即 executor.awaitTermination 的 timeout 参数
小结
参数类别 | 参数值 | jedisConfig 值 | commons-pool 默认值 | 解释 |
---|---|---|---|---|
根本 | lifo | true | true | 后进先出 |
根本 | fairness | false | false | 非偏心机制 |
数量 | maxTotal | 8 | – | 最大连接数 |
数量 | maxIdle | 8 | – | 最大空间连接数 |
数量 | minIdle | 0 | – | 起码闲暇连接数 |
阻塞获取 | blockWhenExhausted | true | true | 连接池耗尽时,获取连贯是否阻塞期待 |
阻塞获取 | maxWaitMillis | -1 | -1 | blockWhenExhausted 为 true 时阻塞期待多久 |
衰弱检测 | testOnCreate | fasle | fasle | 创立连贯的时候检测 |
衰弱检测 | testOnBorrow | false | false | 借用连贯的时候检测 |
衰弱检测 | testOnReturn | false | false | 偿还连贯的时候检测 |
衰弱检测 | testWhileIdle | true | false | 在闲暇连贯检测时检测 |
evict | durationBetweenEvictionRuns | 30 秒 | -1 | evictor 线程运行距离,- 1 为不运行 |
evict | minEvictableIdleDuration | 60 秒 | 30 分钟 | 连贯闲暇的最小工夫 |
evict | softMinEvictableIdleDuration | -1 | -1 | - 1 即 Long.MAX_VALUE,它与 idleCount>minIdle 条件一起思考 |
evict | numTestsPerEvictionRun | -1 | 3 | 每次闲暇连贯回收器线程 (如果有) 运行时查看的连贯数量, - 1 示意全副 |
evict | evictionPolicyClassName | DefaultEvictionPolicy | DefaultEvictionPolicy | 判断是否须要 evict 的 PolicyClass |
evict | evictorShutdownTimeoutDuration | 10s | 10s | 敞开 evictor 线程池的等待时间 |
jmx | jmxEnabled | true | true | 是否开启 jmx |
jmx | jmxNamePrefix | pool | pool | jmx 名称前缀 |
jmx | jmxNameBase | null | null | null 示意由 pool 本人定义 jmxNameBase |
JedisConfig 默认帮咱们配置了 testWhileIdle 为 true(
默认为 false
),minEvictableIdleTime 为 60s(默认为 30 分钟
),timeBetweenEvictionRuns 为 30s(默认为 -1
),numTestsPerEvictionRun 为 -1(即检测所有闲暇连贯,默认值为 3
),利用 evictor 线程来检测闲暇连贯的衰弱状况另外由聊聊 jedis 的 return 行为这篇剖析能够得悉在执行命令时若 redis 出问题,Jedis 自身会标记底层 connection 为 broken,在 finally 偿还时会 destory 连贯,保障连接池连贯最终都会被清空重建。
另外在并发量比拟大的场景,若要保障连接池的稳固数量则能够把 minIdle 设置成与 maxTotal 和 maxIdle 一样即可;若不想因为 idle 工夫被频繁 destory 则能够设置 minEvictableIdleTime 为 -1,evict 办法始终返回 false,evictor 线程始终走的是 testWhileIdle 的逻辑;maxWaitMillis 要设置一个正当值防止连接池耗尽阻塞线程,或者间接设置 blockWhenExhausted 为 false
doc
- GenericObjectPool 参数解析
- JedisPool 资源池优化
- 一次拜访 Redis 延时高问题排查与总结
- 实战总结|一次拜访 Redis 延时高问题排查与总结(续)