共计 12168 个字符，预计需要花费 31 分钟才能阅读完成。
序
本文主要研究一下 jdk httpclient 的 retry 参数
DEFAULT_MAX_ATTEMPTS
java.net.http/jdk/internal/net/http/MultiExchange.java
class MultiExchange<T> {
static final Logger debug =
Utils.getDebugLogger(“MultiExchange”::toString, Utils.DEBUG);
private final HttpRequest userRequest; // the user request
private final HttpRequestImpl request; // a copy of the user request
final AccessControlContext acc;
final HttpClientImpl client;
final HttpResponse.BodyHandler<T> responseHandler;
final HttpClientImpl.DelegatingExecutor executor;
final AtomicInteger attempts = new AtomicInteger();
HttpRequestImpl currentreq; // used for retries & redirect
HttpRequestImpl previousreq; // used for retries & redirect
Exchange<T> exchange; // the current exchange
Exchange<T> previous;
volatile Throwable retryCause;
volatile boolean expiredOnce;
volatile HttpResponse<T> response = null;
// Maximum number of times a request will be retried/redirected
// for any reason
static final int DEFAULT_MAX_ATTEMPTS = 5;
static final int max_attempts = Utils.getIntegerNetProperty(
“jdk.httpclient.redirects.retrylimit”, DEFAULT_MAX_ATTEMPTS
);
//……
}
这里有一个 AtomicInteger 类型的 attempts 变量，用来记录请求次数
另外还有一个 max_attempts，读取的是 jdk.httpclient.redirects.retrylimit 值，读取不到默认取 DEFAULT_MAX_ATTEMPTS，为 5
MultiExchange.responseAsyncImpl
java.net.http/jdk/internal/net/http/MultiExchange.java
private CompletableFuture<Response> responseAsyncImpl() {
CompletableFuture<Response> cf;
if (attempts.incrementAndGet() > max_attempts) {
cf = failedFuture(new IOException(“Too many retries”, retryCause));
} else {
if (currentreq.timeout().isPresent()) {
responseTimerEvent = ResponseTimerEvent.of(this);
client.registerTimer(responseTimerEvent);
}
try {
// 1. apply request filters
// if currentreq == previousreq the filters have already
// been applied once. Applying them a second time might
// cause some headers values to be added twice: for
// instance, the same cookie might be added again.
if (currentreq != previousreq) {
requestFilters(currentreq);
}
} catch (IOException e) {
return failedFuture(e);
}
Exchange<T> exch = getExchange();
// 2. get response
cf = exch.responseAsync()
.thenCompose((Response response) -> {
HttpRequestImpl newrequest;
try {
// 3. apply response filters
newrequest = responseFilters(response);
} catch (IOException e) {
return failedFuture(e);
}
// 4. check filter result and repeat or continue
if (newrequest == null) {
if (attempts.get() > 1) {
Log.logError(“Succeeded on attempt: ” + attempts);
}
return completedFuture(response);
} else {
this.response =
new HttpResponseImpl<>(currentreq, response, this.response, null, exch);
Exchange<T> oldExch = exch;
return exch.ignoreBody().handle((r,t) -> {
previousreq = currentreq;
currentreq = newrequest;
expiredOnce = false;
setExchange(new Exchange<>(currentreq, this, acc));
return responseAsyncImpl();
}).thenCompose(Function.identity());
} })
.handle((response, ex) -> {
// 5. handle errors and cancel any timer set
cancelTimer();
if (ex == null) {
assert response != null;
return completedFuture(response);
}
// all exceptions thrown are handled here
CompletableFuture<Response> errorCF = getExceptionalCF(ex);
if (errorCF == null) {
return responseAsyncImpl();
} else {
return errorCF;
} })
.thenCompose(Function.identity());
}
return cf;
}
进入该方法的时候，调用 attempts.incrementAndGet()，递增请求次数，然后判断有无超出限制，有则返回带有 new IOException(“Too many retries”, retryCause) 异常的 failedFuture，即通过 CompletableFuture.completeExceptionally 返回
如果没有超出限制，但是执行请求失败，则调用 getExceptionalCF 来判断是否应该重试，如果返回 null，则重试，通过再次调用 responseAsyncImpl，通过这种递归调用完成重试逻辑
MultiExchange.getExceptionalCF
java.net.http/jdk/internal/net/http/MultiExchange.java
/**
* Takes a Throwable and returns a suitable CompletableFuture that is
* completed exceptionally, or null.
*/
private CompletableFuture<Response> getExceptionalCF(Throwable t) {
if ((t instanceof CompletionException) || (t instanceof ExecutionException)) {
if (t.getCause() != null) {
t = t.getCause();
}
}
if (cancelled && t instanceof IOException) {
if (!(t instanceof HttpTimeoutException)) {
t = toTimeoutException((IOException)t);
}
} else if (retryOnFailure(t)) {
Throwable cause = retryCause(t);
if (!(t instanceof ConnectException)) {
if (!canRetryRequest(currentreq)) {
return failedFuture(cause); // fails with original cause
}
}
// allow the retry mechanism to do its work
retryCause = cause;
if (!expiredOnce) {
if (debug.on())
debug.log(t.getClass().getSimpleName() + ” (async): retrying…”, t);
expiredOnce = true;
// The connection was abruptly closed.
// We return null to retry the same request a second time.
// The request filters have already been applied to the
// currentreq, so we set previousreq = currentreq to
// prevent them from being applied again.
previousreq = currentreq;
return null;
} else {
if (debug.on()) {
debug.log(t.getClass().getSimpleName()
+ ” (async): already retried once.”, t);
}
t = cause;
}
}
return failedFuture(t);
}
private boolean retryOnFailure(Throwable t) {
return t instanceof ConnectionExpiredException
|| (RETRY_CONNECT && (t instanceof ConnectException));
}
/** Returns true if the given request can be automatically retried. */
private static boolean canRetryRequest(HttpRequest request) {
if (RETRY_ALWAYS)
return true;
if (isIdempotentRequest(request))
return true;
return false;
}
/** Returns true is given request has an idempotent method. */
private static boolean isIdempotentRequest(HttpRequest request) {
String method = request.method();
switch (method) {
case “GET” :
case “HEAD” :
return true;
default :
return false;
}
}
private Throwable retryCause(Throwable t) {
Throwable cause = t instanceof ConnectionExpiredException ? t.getCause() : t;
return cause == null ? t : cause;
}
/** True if ALL (even non-idempotent) requests can be automatic retried. */
private static final boolean RETRY_ALWAYS = retryPostValue();
/** True if ConnectException should cause a retry. Enabled by default */
private static final boolean RETRY_CONNECT = retryConnect();
private static boolean retryPostValue() {
String s = Utils.getNetProperty(“jdk.httpclient.enableAllMethodRetry”);
if (s == null)
return false;
return s.isEmpty() ? true : Boolean.parseBoolean(s);
}
private static boolean retryConnect() {
String s = Utils.getNetProperty(“jdk.httpclient.disableRetryConnect”);
if (s == null)
return false;
return s.isEmpty() ? true : Boolean.parseBoolean(s);
}
如果 cancelled 为 true 且是 IOException 则直接返回，否则先判断 retryOnFailure 再判断 canRetryRequest(如果不是 ConnectException 才走 canRetryRequest 这个判断)
retryOnFailure 方法判断如果是 ConnectionExpiredException 或者是 ConnectException 且开启 retryConnect，则返回 true
RETRY_CONNECT 读取的是 jdk.httpclient.disableRetryConnect 参数，如果值为 null，则方法返回 false，即不进行 retryConnect
canRetryRequest 首先判断 RETRY_ALWAYS，在判断 isIdempotentRequest(GET、HEAD 方法才重试)，都不是则返回 false
RETRY_ALWAYS 读取的是 jdk.httpclient.enableAllMethodRetry，如果值为 null，则方法返回 false，即不进行 retryPostValue
如果该重试的话，则返回 null，responseAsyncImpl 里头在 getExceptionalCF 返回 null 的时候，重新调用了一次 responseAsyncImpl，通过递归调用来完成重试逻辑
NetProperties
java.base/sun/net/NetProperties.java
public class NetProperties {
private static Properties props = new Properties();
static {
AccessController.doPrivileged(
new PrivilegedAction<Void>() {
public Void run() {
loadDefaultProperties();
return null;
}});
}
private NetProperties() {};
/*
* Loads the default networking system properties
* the file is in jre/lib/net.properties
*/
private static void loadDefaultProperties() {
String fname = StaticProperty.javaHome();
if (fname == null) {
throw new Error(“Can’t find java.home ??”);
}
try {
File f = new File(fname, “conf”);
f = new File(f, “net.properties”);
fname = f.getCanonicalPath();
InputStream in = new FileInputStream(fname);
BufferedInputStream bin = new BufferedInputStream(in);
props.load(bin);
bin.close();
} catch (Exception e) {
// Do nothing. We couldn’t find or access the file
// so we won’t have default properties…
}
}
/**
* Get a networking system property. If no system property was defined
* returns the default value, if it exists, otherwise returns
* <code>null</code>.
* @param key the property name.
* @throws SecurityException if a security manager exists and its
* <code>checkPropertiesAccess</code> method doesn’t allow access
* to the system properties.
* @return the <code>String</code> value for the property,
* or <code>null</code>
*/
public static String get(String key) {
String def = props.getProperty(key);
try {
return System.getProperty(key, def);
} catch (IllegalArgumentException e) {
} catch (NullPointerException e) {
}
return null;
}
/**
* Get an Integer networking system property. If no system property was
* defined returns the default value, if it exists, otherwise returns
* <code>null</code>.
* @param key the property name.
* @param defval the default value to use if the property is not found
* @throws SecurityException if a security manager exists and its
* <code>checkPropertiesAccess</code> method doesn’t allow access
* to the system properties.
* @return the <code>Integer</code> value for the property,
* or <code>null</code>
*/
public static Integer getInteger(String key, int defval) {
String val = null;
try {
val = System.getProperty(key, props.getProperty(key));
} catch (IllegalArgumentException e) {
} catch (NullPointerException e) {
}
if (val != null) {
try {
return Integer.decode(val);
} catch (NumberFormatException ex) {
}
}
return defval;
}
/**
* Get a Boolean networking system property. If no system property was
* defined returns the default value, if it exists, otherwise returns
* <code>null</code>.
* @param key the property name.
* @throws SecurityException if a security manager exists and its
* <code>checkPropertiesAccess</code> method doesn’t allow access
* to the system properties.
* @return the <code>Boolean</code> value for the property,
* or <code>null</code>
*/
public static Boolean getBoolean(String key) {
String val = null;
try {
val = System.getProperty(key, props.getProperty(key));
} catch (IllegalArgumentException e) {
} catch (NullPointerException e) {
}
if (val != null) {
try {
return Boolean.valueOf(val);
} catch (NumberFormatException ex) {
}
}
return null;
}
}
这里通过 loadDefaultProperties 先加载默认配置，读取的是 JAVA_HOME/conf/net.properties 文件
然后 getString、getInteger、getBoolean 方法采用的是 System.getProperty 来读取，而 net.properties 值仅仅作为 System.getProperty 的 defaultValue
因此要设置 httpclient 相关参数，只需要通过 System.setProperty 或者 - D 来设置即可
net.properties
/Library/java/JavaVirtualMachines/jdk-11.jdk/Contents/home/conf/net.properties
java.net.useSystemProxies=false
http.nonProxyHosts=localhost|127.*|[::1]
ftp.nonProxyHosts=localhost|127.*|[::1]
jdk.http.auth.tunneling.disabledSchemes=Basic
net.properties 文件默认设置了如上四个参数
相关异常
HttpTimeoutException
java.net.http/java/net/http/HttpTimeoutException.java
/**
* Thrown when a response is not received within a specified time period.
*
* @since 11
*/
public class HttpTimeoutException extends IOException {
private static final long serialVersionUID = 981344271622632951L;
/**
* Constructs an {@code HttpTimeoutException} with the given detail message.
*
* @param message
* The detail message; can be {@code null}
*/
public HttpTimeoutException(String message) {
super(message);
}
}
属于 java.net.http 包，继承至 IOException
如果设置了 request 的 timeout，则注册 ResponseTimerEvent，在超时时抛出 HttpTimeoutException: request timed out，同时设置 MultiExchange 的 cancelled 为 true
这类由于客户端设置超时引起的 HttpTimeoutException，不会进行重试，即使开启相关重试参数
如果这个时间设置得太短，则在 connect 的时候就超时了，这个时候会抛出 HttpConnectTimeoutException，而非 HttpTimeoutException: request timed out
HttpConnectTimeoutException
java.net.http/java/net/http/HttpConnectTimeoutException.java
/**
* Thrown when a connection, over which an {@code HttpRequest} is intended to be
* sent, is not successfully established within a specified time period.
*
* @since 11
*/
public class HttpConnectTimeoutException extends HttpTimeoutException {
private static final long serialVersionUID = 321L + 11L;
/**
* Constructs an {@code HttpConnectTimeoutException} with the given detail
* message.
*
* @param message
* The detail message; can be {@code null}
*/
public HttpConnectTimeoutException(String message) {
super(message);
}
}
属于 java.net.http 包，继承至 HttpTimeoutException
如果设置了 client 的 connectTimeout，则会注册 ConnectTimerEvent，在超时时抛出 ConnectException(“HTTP connect timed out”)，同时设置 MultiExchange 的 cancelled 为 true，这个在 MultiExchange.getExceptionalCF 方法里头会被包装为 HttpConnectTimeoutException
ConnectionExpiredException
java.net.http/jdk/internal/net/http/common/ConnectionExpiredException.java
/**
* Signals that an end of file or end of stream has been reached
* unexpectedly before any protocol specific data has been received.
*/
public final class ConnectionExpiredException extends IOException {
private static final long serialVersionUID = 0;
/**
* Constructs a {@code ConnectionExpiredException} with a detail message of
* “subscription is finished” and the given cause.
*
* @param cause the throwable cause
*/
public ConnectionExpiredException(Throwable cause) {
super(“subscription is finished”, cause);
}
}
一般是在 read error 的时候触发，比如 EOFException，IOException(“connection reset by peer)，或者 SSLHandshakeException
小结
jdk httpclient 的 retry 参数涉及到的参数如下：
jdk.httpclient.redirects.retrylimit(默认为 5，用来控制重试次数，不过实际上还有 expiredOnce 参数，看代码貌似顶多重试一次)
jdk.httpclient.disableRetryConnect(默认为 null，即 RETRY_CONNECT 为 false，不在 ConnectException 的时候 retry)
jdk.httpclient.enableAllMethodRetry(默认为 null，即 RETRY_ALWAYS 为 false，即需要判断请求方法是否幂等来决定是否重试)
是否重试的判断逻辑如下：
如果重试次数超过限制，则返回失败，否则往下
如果 cancelled 为 true(这里如果 request 设置了 timeout，触发时 cancelled 设置为 true) 且是 IOException(例如设置了连接超时抛出的 HttpConnectTimeoutException)，则不走重试逻辑；否则往下
如果 retryOnFailure(ConnectionExpiredException，或者 ConnectException 且开启 retryConnect)，则往下
如果是异常不是 ConnectException，则还额外判断 canRetryRequest(判断该请求类型是否允许重试)，满足则往下
如果 expiredOnce 为 false，则返回 null，即满足重试条件，走递归重试
doc
HttpClient javadoc
