共计 5751 个字符,预计需要花费 15 分钟才能阅读完成。
明天群里有位大佬,发了个链接 http://93.175.29.89:8008/,说爬这个网址的时候,IO 会始终卡在那,始终没有返回响应。
那个网址是他结构的一个非凡申请,输入一个视频流,然而服务器端不返回 Content-Length,也不输入实在数据,就是输入不到 1024 字节的流后就始终停在那也不 close,浏览器关上的成果就是看到了视频的前几帧,而后始终卡在哪转圈。
这么说来,感觉不是个大问题,设置下 ReadTimeout 不就好了么,大佬说他也设置了,然而有效,他应用的 python 代码实现,刚开始我感觉是他代码的问题,或者那个 API 库实现的问题,就用 Java 也实现了一把
package sms.bai.util;
import com.squareup.okhttp.Headers;
import com.squareup.okhttp.OkHttpClient;
import com.squareup.okhttp.Request;
import com.squareup.okhttp.Response;
import java.io.IOException;
import java.util.concurrent.*;
public class Req {public static void reqUrl() throws IOException {OkHttpClient client = new OkHttpClient();
client.setConnectTimeout(5,TimeUnit.SECONDS);
client.setReadTimeout(5,TimeUnit.SECONDS);
Request request = new Request.Builder()
.url("http://93.175.29.89:8008/")
.build();
Response response = client.newCall(request).execute();
if (!response.isSuccessful()) {throw new IOException("服务器端谬误:" + response);
}
Headers responseHeaders = response.headers();
for (int i = 0; i < responseHeaders.size(); i++) {System.out.println(responseHeaders.name(i) + ":" + responseHeaders.value(i));
}
System.out.println(response.body().string());
}
public static void main(String[] args) throws IOException {reqUrl();
}
}
果然如其所言,无论设置 ConnectTimeout 还是 ReadTimeout 都是有效的,代码始终停留在输入那里,不输入任何 body(浏览器里还能勉强看到画面),程序也不 stop
Content-Type: multipart/x-mixed-replace;boundary=---nessy2jpegboundary
OkHttp-Sent-Millis: 1582028133591
OkHttp-Received-Millis: 1582028133875
这里用的是 OkHttp 库 , 换其它库或者用 Java 自带的 HttpUrlConnection 实践上成果也是一样的。
用 ffmpeg 来看看这个申请
[kk@kk ~]$ ffmpeg -i http://93.175.29.89:8008/ -f mp4 out.mp4
ffmpeg version n4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9.2.0 (GCC)
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input #0, mpjpeg, from 'http://93.175.29.89:8008/':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x562ad6812cc0] using SAR=1/1
[libx264 @ 0x562ad6812cc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x562ad6812cc0] profile High, level 3.0, 4:2:0, 8-bit
[libx264 @ 0x562ad6812cc0] 264 - core 159 r2991 1771b55 - H.264/MPEG-4 AVC codec - Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'out.mp4':
Metadata:
encoder : Lavf58.29.100
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc), 640x480 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc
Metadata:
encoder : Lavc58.54.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame= 83 fps=1.1 q=-1.0 Lsize= 336kB time=00:00:03.20 bitrate= 859.8kbits/s speed=0.0436x
ffmpeg 能辨认出是一个视频流,然而会始终卡在 frame=xx 这里,始终读取帧而不进行。强行终止后能输入一个时长有几秒的视频
解决方案
看来依附 HttpUrlConnection 中的 SocketTimeoutException 是无解了,只能在里面套一层了。main 办法改成如下
public static void main(String[] args) throws Exception {final ExecutorService exec = Executors.newFixedThreadPool(1);
Callable<String> call = new Callable<String>() {public String call() throws Exception {
// 开始执行耗时操作
reqUrl();
return "线程执行实现.";
}
};
Future<String> future = null;
try {future = exec.submit(call);
String obj = future.get(1000 * 10, TimeUnit.MILLISECONDS); // 工作解决超时工夫设为 10 秒
System.out.println("工作胜利返回:" + obj);
} catch (TimeoutException ex) {System.out.println("解决超时啦....");
ex.printStackTrace();
future.cancel(true);
} catch (Exception e) {System.out.println("解决失败.");
e.printStackTrace();}finally {
// 敞开线程池
System.out.println("敞开线程池");
exec.shutdown();}
}
这下能失去冀望的后果了
Content-Type: multipart/x-mixed-replace;boundary=---nessy2jpegboundary
OkHttp-Sent-Millis: 1582028854911
OkHttp-Received-Millis: 1582028855178
解决超时啦....
java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at sms.bai.util.Req.main(Req.java:47)
敞开线程池
Process finished with exit code 0
那这个 HttpUrlConnection 里的超时到底是啥意思呢?为什么有效呢?看一下文档。
ConnectTimeout , java 是这样解释的:
Sets a specified timeout value, in milliseconds, to be used when opening a communications link to the resource referenced by this URLConnection. If the timeout expires before the connection can be established, a java.net.SocketTimeoutException is raised. A timeout of zero is interpreted as an infinite timeout.
Some non-standard implmentation of this method may ignore the specified timeout. To see the connect timeout set, please call getConnectTimeout().
意思是用来建设连贯的工夫。如果到了指定的工夫,还没建设连贯,则报异样。这个比拟好了解。
ReadTimeout , Java 是这样解释的:
Sets the read timeout to a specified timeout, in milliseconds. A non-zero value specifies the timeout when reading from Input stream when a connection is established to a resource. If the timeout expires before there is data available for read, a java.net.SocketTimeoutException is raised. A timeout of zero is interpreted as an infinite timeout.
Some non-standard implementation of this method ignores the specified timeout. To see the read timeout set, please call getReadTimeout().
意思是曾经建设连贯,并开始读取服务端资源。如果到了指定的工夫,没有可能的数据被客户端读取,则报异样。
也就是说 setReadTimeout not mean read complete, it mean when wait for 10s, when there’re no more data read in, will throw a timeoutexception。
所以针对这种非凡的服务器结构的异样流,是没法用 SocketTimeoutException 来解决超时的,只能在里面再设置一层,通过线程的超时来管制。
另外提一句,python 是通过设置 gevent 超时来解决的,原理是一样的。
扩大与思考
Callable 和 Feature 原本是 Java 里线程池里的概念,然而这里却被用来解决超时问题,甚至是某些场景下必须的工具。线程池还有哪些妙用呢?
还有哪些场景须要用到 Callable 和 Feature 呢?
对于某些不欠缺的连贯库,没有提供 ConnectTimeout 和 ReadTimeout 的机制,咱们就能用上线程池的超时机制。比方我在应用 hdfs 的官网原生 Java client 时,它的连贯办法参数里并没有超时设置,那么如果 HDFS 服务挂了后,客户端也只能傻傻期待,这种状况下,本人实现一套超时机制就很有必要了。