In recent years, the technology application of RDMA network cards has caused a whirlwind in the global Ethernet communications market. In particular, some Internet giants such as Ali, Tencent, Inspur, Supermicro, Lenovo and other Internet giants have deployed a large number of RDMA network cards on their servers. In 2019, according to official data reports, Tmall’s daily turnover on Double 11 reached 268.4 billion yuan, the peak order innovation reached 544,000 transactions per second, and the single-day data processing volume reached 970PB. This shocking value appeared once again. The application of RDMA network card has pushed the topic to the top.

However, many users said that looking at this data is very impressive, but how the RDMA network card is used in the actual environment is at a loss. In fact, the actual application of RDMA network cards is not as mysterious as everyone thinks. The emergence of RDMA network cards is actually to solve the server-side data processing delay. In other words, the mission of the RDMA network card is to reduce the CPU load. , Improve network throughput and reduce network delay.

1 About RDMA

RDMA (Thel name: Remote Direct Memory Access), which means remote direct memory access, is a network communication protocol that was first applied in the field of high-performance computing, and has gradually become popular in data centers. RDMA allows user programs to bypass the operating system kernel (CPU) and directly interact with the network card for network communication, thereby providing high bandwidth and extremely small latency. Therefore, the working principle of the RDMA network card is that, compared with the traditional TCP/CP network card, the RDMA network card omits the participation of the core (CPU), and all data transfers directly from the application to the network card.

2 Three network protocols of RDMA

Currently, RDMA generally has three network protocols: Infiniband, RoCE, and iWARP

1. Infiniband is a network protocol tailored specifically for RDMA, which can ensure the reliability of data transmission from the hardware level. Although InfiniBand technical specifications and standard specifications were officially published in 2000, InfiniBand Architecture (IBA) has been widely used on cluster supercomputers after 2005. The biggest reason for the slow development is that Infiniband requires its own dedicated hardware from L2 to L4. The cost of the enterprise is very high. Now the main InfiniBand network manufacturer is Mellanox.

2. RoCE is a network protocol that allows RDMA to be implemented on Ethernet. It can be divided into RoCE v1 and RoCE v2. Many netizens believe that the architecture of RoCE itself is grafted from InfiniBand, because the lower network header of RoCE is the Ethernet header , The higher web header is the InfiniBand header. It is a low-cost InfiniBand network launched by Mellanox (supporting RoCE v1) to meet market demand.

RoCE networks require lossless Ethernet to achieve low-latency operation, which means that Ethernet switches integrated into the network must support data center bridging (DCB) and priority flow control (PFC) mechanisms in order to maintain lossless traffic. However, the difficulty of lossless Ethernet is that configuration in a modern enterprise environment is a complicated process, and scalability will be greatly limited, but this is not a problem for companies with strong financial resources. Secondly, RoCE can directly use the network card for rack servers or hosts.

3. iWARP is a network protocol that allows RDMA to be executed on TCP. Its advantage is that it can run in today's standard TCP/IP network. RDMA can be used only by purchasing a network card that supports iWARP. For those with a slightly lower financial budget It is especially suitable for enterprises. But its disadvantage is that it is slightly worse than RoCE in performance. After all, you get what you pay for. This principle has been applied since ancient times.

3, how to choose RDMA network card brand

As the saying goes, shop around, the same applies to the choice of RDMA network card brands. Looking at the global market, there are now three major brands of RDMA network card manufacturers, namely Marvell, intel, and Mellanox. Among them, Marvell is the acquired Qlogic brand, and Mellanox is acquired by NVIDIA.

How do we choose brand manufacturers on RDMA network cards? First of all, we know that Mellanox’s Infiniband network is the best representative of RDMA network card performance. It can ensure reliability from hardware transmission, but requires that the entire ecological environment must be equipped with dedicated hardware. Although RDMA network cards that support Ethernet have now been released, this The RDMA network card only supports RoCE v1, which requires that the priority flow control (PFC) mechanism must be met on the switch, which is expensive. In this regard, most data center companies have been excluded.

Only RoCE v2 and iWARP are left. Intel is an American company mainly doing CPU processing. RDMA performance (RoCE v2 and iWARP protocols) is already supported on the 800 series Ethernet controllers. RoCE v2 is an upgraded version of RoCE v1. The biggest improvement is to support IP routing. The biggest feature of the iWARP protocol is that it can run on any IP network and no longer limit the compatibility of related hardware supporting equipment. From this point of view, Intel and Mellanox, Intel The RDMA network card is more ecologically compatible and cost-effective.

Marvell, as we said before, is the acquired Qlogic brand. The Qlogic brand has the absolute right to speak in the Fibre Channel market. Many corporate customers around the world have adopted QLogic’s solutions. On RDMA, Qlogic and Intel are equally committed to RoCE and Intel. There is no prejudice in iWARP. Customers can use both RoCE and iWARP protocols on the same adapter, but Marvell has announced that the Ethernet network controller will be discontinued next year. This is a very regrettable news.

4. Chinese companies independently research and develop RDMA network card manufacturers

In addition to the above three giants, there are still many RDMA network card manufacturers with strong independent research and development capabilities in China, such as Shenzhen Lianrui Electronics Co., Ltd. In 2019, Shenzhen Lianrui Electronics Co., Ltd. (hereinafter referred to as: LR-LINK Lianrui) reached a strategic partnership with Marvell. With the strong strength of Qlogic in Fibre Channel, it created an RDMA network card exclusively for the server field. The main products are dual/four-port 10 Gigabit network cards (models: LRES1004PF-2SFP+, LRES1005PF-4SFP+), 25G server network cards, 10 Gigabit and 25G OCP 3.0 network cards, etc. As Marvell Ethernet network controllers will be discontinued next year, currently LR-LINK Lianrui's existing Marvell network card (RDMA function) is more for regular customers to order.

In addition, after June 2020, LR-LINK Lianrui began to independently develop an Ethernet card adapter based on the Intel 800 series. The products that have been normally sold are 100G dual optical port server network cards, 100G single optical port server network cards, 25G dual optical port server network card; currently on the market at the end of the research are 25G four optical port server network card, 25G dual optical port OCP3.0 network card and 100G dual port OCP 3.0 network card, etc., all have low latency and high network throughput in terms of performance It also has excellent features such as high volume, saving CPU overhead, and supports RDMA function.

In the future, with the rapid soaring demand for data center data processing in the enterprise field, LR-LINK will continue to leverage its industry advantages, innovate and develop more network cards with high performance, high throughput, low latency and other characteristics, and always strive to provide customers around the world with excellent performance Professional network card solutions.