Scatter/Gather I/O,翻译过去是扩散 / 汇集 I/O(又称为 Vectored I/O)。是一种能够单次调用中对多个缓冲区进行输出 / 输入的形式,能够把多个缓冲区数据一次写到数据流中,也能够把一个数据流读取到多个缓冲区
应用 Scatter/Gather I/ O 的次要起因是为了高效和不便,能够在一次 I / O 操作中解决多个 Buffer 的读写
各种操作系统都提供了 Scatter/Gather I/ O 的函数库,能够很不便的调用。对于一些高级语言,也是有相应实现的。
Java 中的实现
Java 中也提供了 Scatter/Gather I/ O 的实现,对应的接口为java.nio.channels.ScatteringByteChannel/java.nio.channels.GatheringByteChannel
,都在 java 的 nio 包下。
先来看一下接口定义:
public interface ScatteringByteChannel extends ReadableByteChannel
{public long read (ByteBuffer [] dsts) throws IOException;
public long read (ByteBuffer [] dsts, int offset, int length) throws IOException;
}
public interface GatheringByteChannel extends WritableByteChannel
{public long write(ByteBuffer[] srcs) throws IOException;
public long write(ByteBuffer[] srcs, int offset, int length) throws IOException;
}
应用也很简略,因为 FileChannel/
`SocketChanel 是实现了这两个接口的,也就是说应用
FileChannel/SocketChanel` 时能够间接应用 Scatter/Gather I/O
Java 中的应用例子
上面的例子中创立了两个 ByteBuyffer,一个 ByteBuffer 存储一个随机数,另一个 ByteBuffer 存储一个随机字符串。
第一步,应用GatheringByteChannel
` 将两个 ByteBuffer 中的数据写入至 `FileChannel
第二部,应用 ScatteringByteChannel
将 FileChannel 中的数据读取至两个 ByteBuffer 中
第三步,验证读取后的数据是否匹配
示例代码起源:https://howtodoinjava.com/java/nio/nio-scatter-gather-vectored-io/
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.GatheringByteChannel;
import java.nio.channels.ScatteringByteChannel;
public class ScatteringAndGatheringIOExample
{public static void main(String params[])
{
String data = "Scattering and Gathering example shown in howtodoinjava.com";
gatherBytes(data);
scatterBytes();}
/*
* gatherBytes() reads bytes from different buffers and writes to file
* channel. Note that it uses a single write for both the buffers.
*/
public static void gatherBytes(String data)
{
//First Buffer holds a random number
ByteBuffer bufferOne = ByteBuffer.allocate(4);
//Second Buffer holds data we want to write
ByteBuffer buffer2 = ByteBuffer.allocate(200);
//Writing Data sets to Buffer
bufferOne.asIntBuffer().put(13);
buffer2.asCharBuffer().put(data);
//Calls FileOutputStream(file).getChannel()
GatheringByteChannel gatherer = createChannelInstance("test.txt", true);
//Write data to file
try
{gatherer.write(new ByteBuffer[] {bufferOne, buffer2});
}
catch (Exception e)
{e.printStackTrace();
}
}
/*
* scatterBytes() read bytes from a file channel into a set of buffers. Note that
* it uses a single read for both the buffers.
*/
public static void scatterBytes()
{
//First Buffer holds a random number
ByteBuffer bufferOne = ByteBuffer.allocate(4);
//Second Buffer holds data we want to write
ByteBuffer bufferTwo = ByteBuffer.allocate(200);
//Calls FileInputStream(file).getChannel()
ScatteringByteChannel scatterer = createChannelInstance("test.txt", false);
try
{
//Reading from the channel
scatterer.read(new ByteBuffer[] {bufferOne, bufferTwo});
}
catch (Exception e)
{e.printStackTrace();
}
//Read the buffers seperately
bufferOne.rewind();
bufferTwo.rewind();
int bufferOneContent = bufferOne.asIntBuffer().get();
String bufferTwoContent = bufferTwo.asCharBuffer().toString();
//Verify the content
System.out.println(bufferOneContent);
System.out.println(bufferTwoContent);
}
public static FileChannel createChannelInstance(String file, boolean isOutput)
{
FileChannel fc = null;
try
{if (isOutput) {fc = new FileOutputStream(file).getChannel();} else {fc = new FileInputStream(file).getChannel();}
}
catch (Exception e) {e.printStackTrace();
}
return fc;
}
}
Netty 的实现??
Netty 中提供了 CompositeByteBuf,尽管看起来相似 Scatter/Gather I/O,但和 JDK 自带的齐全不是一回事,Netty 只是在 Java 层面对 Buffer 进行了组合
参考
- Chapter 4. Advanced File I/O –《Linux System Programming 2nd Edition by Robert Love》
- https://en.wikipedia.org/wiki/Vectored_I/O?oldformat=true
- 扩散 / 汇集 IO(scatter/gather)及 iovec 构造体
- Java NIO Vectored IO