明天群里有位大佬,发了个链接http://93.175.29.89:8008/,说爬这个网址的时候,IO会始终卡在那,始终没有返回响应。
那个网址是他结构的一个非凡申请,输入一个视频流,然而服务器端不返回Content-Length,也不输入实在数据,就是输入不到1024字节的流后就始终停在那也不close,浏览器关上的成果就是看到了视频的前几帧,而后始终卡在哪转圈。
这么说来,感觉不是个大问题,设置下ReadTimeout不就好了么,大佬说他也设置了,然而有效,他应用的python代码实现,刚开始我感觉是他代码的问题,或者那个API库实现的问题,就用Java也实现了一把
package sms.bai.util;import com.squareup.okhttp.Headers;import com.squareup.okhttp.OkHttpClient;import com.squareup.okhttp.Request;import com.squareup.okhttp.Response;import java.io.IOException;import java.util.concurrent.*;public class Req { public static void reqUrl() throws IOException { OkHttpClient client = new OkHttpClient(); client.setConnectTimeout(5,TimeUnit.SECONDS); client.setReadTimeout(5,TimeUnit.SECONDS); Request request = new Request.Builder() .url("http://93.175.29.89:8008/") .build(); Response response = client.newCall(request).execute(); if (!response.isSuccessful()) { throw new IOException("服务器端谬误: " + response); } Headers responseHeaders = response.headers(); for (int i = 0; i < responseHeaders.size(); i++) { System.out.println(responseHeaders.name(i) + ": " + responseHeaders.value(i)); } System.out.println(response.body().string()); } public static void main(String[] args) throws IOException { reqUrl(); }}
果然如其所言,无论设置ConnectTimeout还是ReadTimeout都是有效的,代码始终停留在输入那里,不输入任何body(浏览器里还能勉强看到画面),程序也不stop
Content-Type: multipart/x-mixed-replace;boundary=---nessy2jpegboundaryOkHttp-Sent-Millis: 1582028133591OkHttp-Received-Millis: 1582028133875
这里用的是OkHttp库 ,换其它库或者用Java自带的HttpUrlConnection实践上成果也是一样的。
用ffmpeg来看看这个申请
[kk@kk ~]$ ffmpeg -i http://93.175.29.89:8008/ -f mp4 out.mp4ffmpeg version n4.2.2 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 9.2.0 (GCC) libavutil 56. 31.100 / 56. 31.100 libavcodec 58. 54.100 / 58. 54.100 libavformat 58. 29.100 / 58. 29.100 libavdevice 58. 8.100 / 58. 8.100 libavfilter 7. 57.100 / 7. 57.100 libswscale 5. 5.100 / 5. 5.100 libswresample 3. 5.100 / 3. 5.100 libpostproc 55. 5.100 / 55. 5.100Input #0, mpjpeg, from 'http://93.175.29.89:8008/': Duration: N/A, bitrate: N/A Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 1:1 DAR 4:3], 25 tbr, 25 tbn, 25 tbcStream mapping: Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))Press [q] to stop, [?] for help[libx264 @ 0x562ad6812cc0] using SAR=1/1[libx264 @ 0x562ad6812cc0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2[libx264 @ 0x562ad6812cc0] profile High, level 3.0, 4:2:0, 8-bit[libx264 @ 0x562ad6812cc0] 264 - core 159 r2991 1771b55 - H.264/MPEG-4 AVC codec - Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00Output #0, mp4, to 'out.mp4': Metadata: encoder : Lavf58.29.100 Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc), 640x480 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc Metadata: encoder : Lavc58.54.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1frame= 83 fps=1.1 q=-1.0 Lsize= 336kB time=00:00:03.20 bitrate= 859.8kbits/s speed=0.0436x
ffmpeg 能辨认出是一个视频流,然而会始终卡在frame=xx这里,始终读取帧而不进行。强行终止后能输入一个时长有几秒的视频
解决方案
看来依附HttpUrlConnection中的SocketTimeoutException是无解了,只能在里面套一层了。main办法改成如下
public static void main(String[] args) throws Exception { final ExecutorService exec = Executors.newFixedThreadPool(1); Callable<String> call = new Callable<String>() { public String call() throws Exception { //开始执行耗时操作 reqUrl(); return "线程执行实现."; } }; Future<String> future = null; try { future = exec.submit(call); String obj = future.get(1000 * 10, TimeUnit.MILLISECONDS); //工作解决超时工夫设为 10 秒 System.out.println("工作胜利返回:" + obj); } catch (TimeoutException ex) { System.out.println("解决超时啦...."); ex.printStackTrace(); future.cancel(true); } catch (Exception e) { System.out.println("解决失败."); e.printStackTrace(); }finally { // 敞开线程池 System.out.println("敞开线程池"); exec.shutdown(); } }
这下能失去冀望的后果了
Content-Type: multipart/x-mixed-replace;boundary=---nessy2jpegboundaryOkHttp-Sent-Millis: 1582028854911OkHttp-Received-Millis: 1582028855178解决超时啦....java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:205) at sms.bai.util.Req.main(Req.java:47)敞开线程池Process finished with exit code 0
那这个HttpUrlConnection里的超时到底是啥意思呢?为什么有效呢?看一下文档。
ConnectTimeout , java 是这样解释的:
Sets a specified timeout value, in milliseconds, to be used when opening a communications link to the resource referenced by this URLConnection. If the timeout expires before the connection can be established, a java.net.SocketTimeoutException is raised. A timeout of zero is interpreted as an infinite timeout.
Some non-standard implmentation of this method may ignore the specified timeout. To see the connect timeout set, please call getConnectTimeout().
意思是用来建设连贯的工夫。如果到了指定的工夫,还没建设连贯,则报异样。 这个比拟好了解。
ReadTimeout , Java 是这样解释的:
Sets the read timeout to a specified timeout, in milliseconds. A non-zero value specifies the timeout when reading from Input stream when a connection is established to a resource. If the timeout expires before there is data available for read, a java.net.SocketTimeoutException is raised. A timeout of zero is interpreted as an infinite timeout.
Some non-standard implementation of this method ignores the specified timeout. To see the read timeout set, please call getReadTimeout().
意思是曾经建设连贯,并开始读取服务端资源。如果到了指定的工夫,没有可能的数据被客户端读取,则报异样。
也就是说setReadTimeout not mean read complete, it mean when wait for 10s, when there're no more data read in, will throw a timeoutexception。
所以针对这种非凡的服务器结构的异样流,是没法用SocketTimeoutException来解决超时的,只能在里面再设置一层,通过线程的超时来管制。
另外提一句,python是通过设置gevent超时来解决的,原理是一样的。
扩大与思考
Callable和Feature原本是Java里线程池里的概念,然而这里却被用来解决超时问题,甚至是某些场景下必须的工具。线程池还有哪些妙用呢?
还有哪些场景须要用到Callable和Feature呢?
对于某些不欠缺的连贯库,没有提供ConnectTimeout和ReadTimeout的机制,咱们就能用上线程池的超时机制。比方我在应用hdfs的官网原生Java client时,它的连贯办法参数里并没有超时设置,那么如果HDFS服务挂了后,客户端也只能傻傻期待,这种状况下,本人实现一套超时机制就很有必要了。