目录

​infiniband应用到Redis之后的提升效果​

​WAN上InfiniBand传输服务的实验分析​​ 

​RC和UD性能比较​



UC和RC的区别

(https://www.mellanox.com/related-docs/prod_software/RDMA_Aware_Programming_user_manual.pdf)

2.2 Transport Modes

There are several different transport modes you may select from when establishing a QP. Operations

available in each mode are shown below in the table below. RD is not supported by this

API.

【RDMA】infiniband提升Redis性能|UC和RC时延比较|RC和UD性能比较_redis

2.2.1 Reliable Connection (RC)

Queue Pair is associated with only one other QP.

Messages transmitted by the send queue of one QP are reliably delivered to receive queue of the

other QP.

Packets are delivered in order.

A RC connection is very similar to a TCP connection.

2.2.2 Unreliable Connection (UC)

A Queue Pair is associated with only one other QP.

The connection is not reliable so packets may be lost.

Messages with errors are not retried by the transport, and error handling must be provided by a

higher level protocol.

2.2.3 Unreliable Datagram (UD)

A Queue Pair may transmit and receive single-packet messages to/from any other UD QP.

Ordering and delivery are not guaranteed, and delivered packets may be dropped by the receiver.

Multicast messages are supported (one to many).

A UD connection is very similar to a UDP connection.

 

infiniband应用到Redis之后的提升效果

国防科技大学论文:

连接:http://ir.nsfc.gov.cn/paperDownload/ZD2959605.pdf

 

原文:http://ir.nsfc.gov.cn/paperDownload/ZD2959605.pdf

 

测试环境:

x86-64

2.10GHz Intel Xeon E5-2620L 6-core processors, featuring 32KB for L1 instruction and data caches, 256KB L2 and 15MB L3 cache.

16GB.

Mellanox MT27500 ConnectX-3 40 GbpsInfiniBand HCA

Red Hat Enterprise Linux Server 6 ,kernel 2.6.32.

测试工具:

InfiniBand verbs performance test tool

如:ib_send_lat, ib_read_lat, ib_write_lat

测试结果:

Inlined模式下,小于256个字节的小消息可以inline在WR中,以避免RDMA NIC通过DMAread提取小消息。这种方法可以有效地避免DMA操作从而减少延迟。

图1表明:

当消息size 小于256 bytes时,inlined 的SEND/RECV 可以明显的降低时延

不用 inlined 的SEND/RECV 操作在UD 传输时也可以轻微降低时延,但效果不明显。

图2表明,与SEND / RECV操作相似,在传输小消息时,使用inlined的RDMA write在RC传输和UC传输上具有较低的延迟。

图3表明,inlined的RDMA write 在UC传输的时延比RC传输的时延低。

而图4表明,非inlined的RDMA write在UC传输时不比RC传输具有更好的性能。

 

【RDMA】infiniband提升Redis性能|UC和RC时延比较|RC和UD性能比较_redis_02

 

 

WAN上InfiniBand传输服务的实验分析 (RC|UC|WRITE|READ|SEND)

橡树岭国家实验室(美)测试结果

https://www.cs.fsu.edu/~yuw/pubs/2008-NAS-Yu.pdf

 

RC和UD性能比较

《Scalable Network Communication using Unreliable RDMA》https://www.osti.gov/servlets/purl/1671985