TCP粘包/拆包问题

 TCP粘包和拆包示意图如下：




Java 解决粘包拆包_Java 解决粘包拆包



TCP粘包/拆包



假设客户端分别发送了两个数据包D1和D2给服务端，由于服务端一次读到的字节数是不确定的，故可能存在以下4中情况.


 1. 服务端分两次读取到了两个独立的数据包，分别是D1和D2,没有粘包和拆包


 2. 服务端一次收到了连个数据包，D1和D2粘合在一起，被称为TCP粘包


 3. 服务端分别两次读取到了两个数据包，第一次读取到了完整的D1包和D2包的部分内容，第二次读取到了D2包的剩余内容，称之为TCP拆包。


 4. 服务端分两次读取到了两个数据包，第一次读取到了D1包的部分内容D1_1,第二次读取到了D1包的剩余内容D1_2和D2包的整包。如果此时服务端TCP接受滑窗非常小，而数据包D1和D2比较大，很有可能会发生第五种可能，即服务端分多次才能将D1和D2包接收完全，期间分生多次拆包。

粘包问题的解决策略

由于底层的TCP 无法理解上层的的业务数据，所以在底层是无法保证数据包不被拆分和重组的，这个问题只能通过上层的应用协议栈设计来解决，根据业界的主流的协议的解决方案，可以归纳如下：
 1. 消息定长，例如每个报文的大小为固定长度200字节，如果不够，空位补空格；
 2. 在包尾增加回车换行符进行分割，例如FTP协议；
 3. 将消息分为消息头和消息体，消息头中包含表示消息总长度的字段，通常设计思路为消息头的第一个字段使用int32来表示消息的总长度；
 4. 更复杂的应用层协议。

未考虑TCP粘包导致功能异常的案例

 未考虑TCP粘包下的客户端启动，至添加自己实现的普通的写数据的TimeClientHandler。

/**
 * @desc 客户端启动
 */
public class TimeClient {

    public void connect(int port, String host) {
        EventLoopGroup workGroup = new NioEventLoopGroup();
        Bootstrap bootstrap = new Bootstrap();
        bootstrap.group(workGroup)
                .channel(NioSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new TimeClientHandler());
                    }
                });

        ChannelFuture future = bootstrap.connect(host, port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeClient().connect(8080, "127.0.0.1");
    }
}

 TimeClientHandler实现如下：

/**
 * @desc 客户端handler
 */
public class TimeClientHandler extends ChannelInboundHandlerAdapter {

    private int count;

    private byte[] request;

    public TimeClientHandler() {
        request = ("QUERY TIME ORDER" + System.getProperty("line.separator")).getBytes();
    }
    /**
     * @desc 接受的服务端返回数据
     */
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {

        ByteBuf byteBuf = (ByteBuf) msg;
        request = new byte[byteBuf.readableBytes()];
        String body = new String(request, "UTF-8");

        System.out.println("now is:" + body.toString() + "; the counter is" + ++count);

    }
    /**
     * @desc 客户端与服务端channel开启的时候，写数据
     */
    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        ByteBuf message = null;
        for (int i = 0; i < 100; i++) {
            message = Unpooled.buffer(request.length);
            message.writeBytes(request);
            ctx.writeAndFlush(message);
        }
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
        super.exceptionCaught(ctx, cause);
    }
}

 服务端启动类：

/**
 * @desc 服务端启动
 */
public class TimeServer {

    public void bind(int port) {

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workGroup = new NioEventLoopGroup();
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(bossGroup, workGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .childHandler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new TimeServerHandler());
                    }
                });

        ChannelFuture future = bootstrap.bind(port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeServer().bind(8080);
    }
}

 服务端TimeServerHandler实现，接收数据之后的处理逻辑：

/**
 * @desc 服务端处理handler
 */
public class TimeServerHandler extends ChannelInboundHandlerAdapter {

    private int count;

    /**
     * @desc 服务端接受到客户端数据的处理逻辑
     */
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {

        ByteBuf byteBuf = (ByteBuf) msg;
        byte[]  request = new byte[byteBuf.readableBytes()];
        byteBuf.readBytes(request);
        String body = new String(request, "UTF-8")
                .substring(0, request.length - System.getProperty("line.separator").length());
        System.out.println("time server receive order:" + body + ";the count is :" + ++ count);

        String currentTime = "QUERY TIME ORDER".equalsIgnoreCase(body) ? new Date(System.currentTimeMillis()).toString() : "BAD ORDER";

        currentTime = currentTime + System.getProperty("line.separator");
        ByteBuf response = Unpooled.copiedBuffer(currentTime.getBytes());
        ctx.writeAndFlush(response);
    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("client connect !!!!!!");
        super.channelActive(ctx);
    }
}

运行结果如下：

QUERY TIME ORDER
QUERY TIME ORDER
QUERY TIME ORDER
QUERY TIME ORDER
QUE;the count is :1
time server receive order:Y TIME ORDER
QUERY TIME ORDER
QUERY TIME ORDER

 发生粘包现象。”QUERY TIME ORDER”会出现不完整的数据包。

利用LineBasedFrameDecoder解决TCP粘包问题

 LineBasedFrameDecoder其实就是按照换行符分隔字符创的解码器。

/**
 * @desc 基于LineBasedFrameDecoder服务端
 */
public class TimeServer {

    public void bind(int port) {

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workGroup = new NioEventLoopGroup();
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(bossGroup, workGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .childHandler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new LineBasedFrameDecoder(1024))//添加LineBasedFrameDecoder
                                .addLast(new TimeServerHandler());
                    }
                });

        ChannelFuture future = bootstrap.bind(port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeServer().bind(8080);
    }
}

 客户端实现，与前面的几乎相同：

/**
 * @desc 基于LineBasedFrameDecoder客户端
 */
public class TimeClient {

    public void connect(int port, String host) {
        EventLoopGroup workGroup = new NioEventLoopGroup();
        Bootstrap bootstrap = new Bootstrap();
        bootstrap.group(workGroup)
                .channel(NioSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new LineBasedFrameDecoder(1024))
                                .addLast(new TimeClientHandler());
                    }
                });

        ChannelFuture future = bootstrap.connect(host, port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeClient().connect(8080, "127.0.0.1");
    }
}

 运行结果如下：

time server receive order:QUERY TIME ORDE;the count is :1
time server receive order:QUERY TIME ORDE;the count is :2
time server receive order:QUERY TIME ORDE;the count is :3
.......
time server receive order:QUERY TIME ORDE;the count is :50

 完美解决粘包现象。LineBasedFrameDecoder的工作原理是依次遍历ByteBuf中的可读字节，判断是否有”“\n”或者”“\r\n”,如果有，就依次位置为结束位置，从可读索引到结束位置区间的字节组成一行。它是以换行符为结束标志的解码器，支持携带结束符或者不携带结束符两种解码方式，同时支持配置单行的最大长度。如果连读读取到最大长度后仍然没有发现换行符，就会抛出异常，同时忽略掉之前读取到的异常码流。
 StringDecoder的功能非常简单，就是将接受到的对象转换成字符串，然后继续调用后面的handler,LineBasedFrameDecoder + StringDecoder组合就是按行切换的文本解码器，它被设计用来支持TCP的粘包和拆包

利用DelimiterBasedFrameDecoder解决TCP粘包问题

 DelimiterBasedFrameDecoder是一种基于分隔符的编码器，用以处理粘包问题。

public class TimeClient {

    public void connect(int port, String host) {
        EventLoopGroup workGroup = new NioEventLoopGroup();
        Bootstrap bootstrap = new Bootstrap();
        bootstrap.group(workGroup)
                .channel(NioSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ByteBuf delimiter = Unpooled.copiedBuffer("$_".getBytes());//定义使用的分隔符

                        ch.pipeline()
                                .addLast(new DelimiterBasedFrameDecoder(1024, delimiter))//添加DelimiterBasedFrameDecoder
                                .addLast(new StringDecoder())
                                .addLast(new TimeClientHandler());
                    }
                });

        ChannelFuture future = bootstrap.connect(host, port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeClient().connect(8080, "127.0.0.1");
    }
}

 客户端实现。

public class TimeClientHandler extends ChannelInboundHandlerAdapter {

    private int count;


    static final String echo = "hi,welcome to netty.$_";


    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {

        System.out.println("this is" + ++count + " times receive client : {" + msg + "}");


    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        IntStream.range(0,20).forEach(i -> {
            ctx.writeAndFlush(Unpooled.copiedBuffer((echo+i).getBytes()));
        });

    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
        super.exceptionCaught(ctx, cause);
    }
}

 服务端实现。

public class TimeServer {

    public void bind(int port) {

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workGroup = new NioEventLoopGroup();
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(bossGroup, workGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .childHandler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ByteBuf delimiter = Unpooled.copiedBuffer("$_".getBytes());
                        ch.pipeline()
                                .addLast(new DelimiterBasedFrameDecoder(1024, delimiter))
                                .addLast(new StringDecoder())
                                .addLast(new TimeServerHandler());
                    }
                });

        ChannelFuture future = bootstrap.bind(port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeServer().bind(8080);
    }
}
public class TimeServerHandler extends ChannelInboundHandlerAdapter {

    private int count;

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {

        String body = (String) msg;
        System.out.println("this is" + ++count + " times receive client : {" + body + "}");
        body += "$_";

        ByteBuf response = Unpooled.copiedBuffer(body.getBytes());
        ctx.writeAndFlush(response);
    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("client connect !!!!!!");
        super.channelActive(ctx);
    }
}
client connect !!!!!!
this is1 times receive client : {hi,welcome to netty.}
this is2 times receive client : {0hi,welcome to netty.}
this is3 times receive client : {1hi,welcome to netty.}
this is4 times receive client : {2hi,welcome to netty.}
this is5 times receive client : {3hi,welcome to netty.}
this is6 times receive client : {4hi,welcome to netty.}
this is7 times receive client : {5hi,welcome to netty.}
this is8 times receive client : {6hi,welcome to netty.}
this is9 times receive client : {7hi,welcome to netty.}
this is10 times receive client : {8hi,welcome to netty.}
this is11 times receive client : {9hi,welcome to netty.}
this is12 times receive client : {10hi,welcome to netty.}
this is13 times receive client : {11hi,welcome to netty.}
this is14 times receive client : {12hi,welcome to netty.}
this is15 times receive client : {13hi,welcome to netty.}
this is16 times receive client : {14hi,welcome to netty.}
this is17 times receive client : {15hi,welcome to netty.}
this is18 times receive client : {16hi,welcome to netty.}
this is19 times receive client : {17hi,welcome to netty.}
this is20 times receive client : {18hi,welcome to netty.}

 没有发生粘包现象。

利用FixedLengthFrameDecoder解决TCP粘包问题
public class TimeServer {

    public void bind(int port) {

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workGroup = new NioEventLoopGroup();
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(bossGroup, workGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .childHandler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new FixedLengthFrameDecoder(20))
                                .addLast(new StringDecoder())
                                .addLast(new TimeServerHandler());
                    }
                });

        ChannelFuture future = bootstrap.bind(port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeServer().bind(8080);
    }
}
public class TimeServerHandler extends ChannelInboundHandlerAdapter {

    private int count;

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        System.out.println("receive client : {" + msg + "}");
    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("client connect !!!!!!");
        super.channelActive(ctx);
    }
}

 此项测试不做客户端实现，直接telnet localhost 8080

pjx@pjxdeMacBook-Pro:~/project/netty-learning$  telnet localhost 8080
Trying ::1...
Connected to localhost.
Escape character is '^]'.
netty fixedLengthFrameDecoder

服务端接受到固定长度数据

receive client : {netty fixedLengthFra}
总结

 DelimiterBasedFrameDecoder用于使用分隔符结尾的消息进行自动解码，FixedLengthFrameDecoder用于对固定长度的消息进行解码。有了上述两种解码器，再结合其他解码器，可以完成对很多消息的自动解码，而且不需要需要再考虑tcp粘包/拆包导致的读半包问题。