TECHNICAL FIELD

This disclosure relates to the field of computer technologies, and in particular, to a packet sending method and apparatus, and a storage device.

BACKGROUND

A RDMA over Converged Ethernet (RoCE) protocol is a network protocol that allows remote direct memory access (RDMA) over an Ethernet network, and is widely applied to a data transmission system. At present, there are two RoCE versions. RoCE v1 and RoCE v2. RoCE v1 is implemented based on an Ethernet link layer, and allows direct access of any two hosts in a same broadcast domain to memory data. RoCE v2 is implemented by a User Datagram Protocol (UDP) layer in an Ethernet Transmission Control Protocol/Internet Protocol (TCP/IP).

Usually, the data transmission system includes: a switch, a storage device, a server, and at least one virtual machine running on the server. Each virtual machine represents one storage access user. The storage device includes at least one disk. The disk may be an NVM Express (NVMe) disk. A read/write access request of the virtual machine to the storage device is sent, in a packet form, by a network interface card of the server to the storage device by using the switch. The switch may simultaneously receive packets of a plurality of servers in the data transmission system. In this case, the switch needs to forward these packets to the storage device through one or more ports. When detecting that wait duration of a packet in a send queue of any port connected to the storage device exceeds a preset duration, the switch determines that the port is congested. An explicit congestion notification (ECN) flag bit in the packet is changed, so that a changed ECN flag bit indicates that the port of the switch is congested. When the ECN flag bit in the packet received by the storage device indicates congestion, the storage device feeds back a congestion notification packet (CNP) to the server sending the packet. In this case, the server reduces a sending rate of the packet based on a preset rate, thereby controlling data transmission of the data transmission system and avoiding packet congestion. Although the foregoing processing method relieves the network congestion problem of the switch, reducing, at a same ratio, the rate at which the server sends a packet affects service processing capabilities of the server and the virtual machine running in the server, and extends service processing duration. This does not fundamentally resolve the network congestion problem in the data transmission system.

SUMMARY

A packet sending method, packet sending apparatus, and a storage device resolve a problem of network congestion in a data transmission system while ensuring service processing capabilities of a server and a virtual machine running on the server.

According to a first aspect, a packet sending method is applied to a data transmission system. The data transmission system includes a first device and a second device. A packet is transmitted between the first device and the second device based on a remote direct memory access RoCE protocol, the first device establishes a first queue pair (QP), and the second device establishes a first QP′ associated with the first QP. The method includes: first, receiving, by a network interface card of the second device, a packet that is from the first device and that is forwarded by a switch, and obtaining an instant rate and a maximum sending rate of packet sending in the first QP; subsequently, determining an adjustment rate of the first QP based on the instant rate at which the first device sends a packet and the maximum sending rate corresponding to the first QP; and then, sending a congestion notification packet carrying the adjustment rate to the first device, to adjust the rate at which the first device sends the packet.

The instant rate of the first QP is a quantity of packets that are sent between the first QP and the first QP′ within a unit time. The maximum sending rate of the first QP is a preset maximum quantity of packets that are allowed to be sent between the first QP and the first QP′ within the unit time.

Further, the network interface card of the second device may obtain the instant rate of the first QP in any one of the following manners:

In a first manner, the network interface card of the second device counts a quantity of packets that are from the first QP, that are received within a unit time, and that are forwarded by using the switch, to determine the instant rate of the first QP.

Specifically, the second device pre-establishes the first QP′ associated with the first QP. When the network interface card of the second device determines, based on the packet, that a port of the switch is congested, the network interface card counts the quantity of packets that belong to the first QP′ and that are received within the unit time, to determine an instant rate at which the network interface card receives the packet sent by the first device, to serve as the instant rate of the first QP.

In a second manner, the first device periodically sends a rate notification packet to the second device.

Specifically, the first device periodically sends the rate notification packet to the network interface card of the second device, and the rate notification packet carries the instant rate at which the first QP sends a packet.

The network interface card of the second device may determine the maximum sending rate of the first QP in the following manner: determining the maximum sending rate of the first QP based on a type of service corresponding to the first QP.

Specifically, maximum sending rates corresponding to a QP are different due to different types of service processed by the QP. When the first QP is established between the first device and the second device, a mapping table between a type of service and a maximum sending rate may be pre-stored in the second device. When a packet is received, the maximum sending rate of the first QP may be determined based on a type of service processed by the first QP.

The adjustment rate of the first device is determined by using an instant rate of a QP and a maximum rate of a QP. In this way, a network congestion problem of the switch can be relieved, and moreover, service processing duration can be reduced while ensuring service processing capabilities of the first device and a virtual machine running on the first device.

In a possible implementation, the packet sending method further includes: when determining the adjustment rate of the first QP, in addition to the foregoing method, determining, by the network interface card of the second device, the adjustment rate by considering a priority of the first device based on a service requirement, to be specific, adjusting, based on the instant rate of the first QP, the maximum sending rate of the first QP, and the priority of the first device, a rate at which the first QP sends a packet. Different weights may be set based on different priorities of the first device in the data transmission system, to preferentially ensure that the switch processes a packet of a QP in the first device having a high priority.

In a possible implementation, when determining the adjustment rate of the first QP, the network interface card of the second device may further determine the adjustment rate of the first QP by considering a priority of the virtual machine on the first device. During specific implementation, a QP in the first device may establish a binding relationship with a specified virtual machine. In this case, all packets of the virtual machine are sent to the second device by using the QP bound to the virtual machine. The network interface card of the second device determines the adjustment rate of the first QP based on the instant rate of the first QP, the maximum sending rate of the first QP, and the service priority of the virtual machine on the first device.

When determining the adjustment rate of the first QP, the network interface card of the second device may determine the adjustment rate of the first QP by simultaneously considering the priorities of the first device and the virtual machine on the first device. When adjusting the rate at which the first QP sends a packet, the network interface card determines the adjustment rate of the first QP not only by considering the instant rate of the first QP and the maximum sending rate of the first QP, but also by considering different service priorities of the first device and the virtual machine on the first device, thereby preferentially ensuring a processing process of a service having a high priority while relieving network congestion, and further improving reasonableness of congestion relief of the data transmission system.

In another possible implementation, the data transmission system further includes at least one third device, each of the at least one third device establishes at least one QP between the second device and each of the at least one third device based on the RoCE protocol, and the data transmission system includes m QPs; and the determining, by the network interface card of the second device, an adjustment rate of the first QP based on the instant rate of the first QP and the maximum sending rate of the first QP includes:

When adjusting the rate at which the first device sends a packet, the network interface card of the second device may obtain the adjustment rate S1of the first QP by using a formula

S1=D⁢A1/C1∑i=1m⁢Ai/Ci
and based on the instant rate A1of the first QP, the maximum sending rate C1of the first QP, and a rate D during congestion of the switch, and the adjustment rate S1of the first QP is the adjustment rate at which the first device sends a packet, where Aiindicates an instant rate of an ithQP of the m QPs, Ciindicates a maximum sending rate of an ithQP in each QP, m is a positive integer, and i is a positive integer ranging from 1 to m.

A capability of forwarding a packet of the first QP by the switch during congestion of the switch can be determined by using the foregoing formula. In this way, the adjustment rate of the first QP is determined, and network congestion of the switch is relieved.

In another possible implementation, the data transmission system further includes at least one third device, each of the at least one third device establishes at least one QP between the second device and each of the at least one third device based on the RoCE protocol, and the data transmission system includes m QPs; and the determining, by the network interface card of the second device, the adjustment rate of the first QP based on the instant rate of the first QP, a priority of the first device, and the maximum sending rate of the first QP includes: when adjusting the rate at which the first device sends a packet, obtaining, by the network interface card of the second device, an adjustment basis by using a formula

S1=DB1⁢A1/C1∑i=1m⁢Ai/Ci
and based on the instant rate A1of the first QP, a weight coefficient B1corresponding to the priority of the first device, the maximum sending rate C1of the first QP, and a rate D during congestion of the switch, where the obtained adjustment rate S1of the first QP is the adjustment basis of the rate at which the first device sends a packet, Aiindicates an instant rate of an ithQP of the m QPs, C indicates a maximum sending rate of the ithQP of the m QPs, m is a positive integer, and i is a positive integer ranging from 1 to m.

In another possible implementation, before the obtaining, by the network interface card of the second device, an instant rate of the first QP, the packet sending method includes: when receiving the packet sent by the first device, reading, by the network interface card of the second device, a flag bit that is in the packet and that is used to indicate whether network congestion occurs on the switch; and when the flag bit indicates that network congestion occurs on the switch, performing the method according to any possible implementation of the first aspect, to adjust the rate at which the first device sends the packet.

An operating status of the data transmission system can be recognized rapidly by reading the flag bit that is in the packet and that indicates whether network congestion occurs on the switch, to adjust the rate at which the first device sends the packet in a timely manner.

According to a second aspect, a packet sending apparatus is configured to perform the method according to the first aspect or any possible implementation of the first aspect, and their technical effects are the same. Specifically, the apparatus includes modules configured to perform the method according to the first aspect or any possible implementation of the first aspect.

In a possible implementation, a packet is transmitted between the packet sending apparatus and a first device based on a RoCE protocol, the first device establishes a first queue pair QP, the packet sending apparatus establishes a first QP′ associated with the first QP, and the packet sending apparatus includes an obtaining module, a processing module, and a sending module, where the obtaining module is configured to: receive a first packet that is from the first QP of the first device and that is forwarded by using a switch, and obtain an instant rate of the first QP and a maximum sending rate of the first QP, where the instant rate of the first QP represents a quantity of packets that are sent between the first QP and the first QP′ within a unit time, and the maximum sending rate of the first QP represents a preset maximum quantity of packets that are allowed to be sent between the first QP and the first QP′ within the unit time; the processing module is configured to determine an adjustment rate of the first QP based on the instant rate of the first QP and the maximum sending rate of the first QP; and the sending module is configured to send a congestion notification packet to the first device, where the congestion notification packet carries the adjustment rate of the first QP.

The apparatus may determine the adjustment rate by using an instant rate of a QP and a maximum rate of a QP. In this way, a network congestion problem of the switch can be relieved, and moreover, service processing duration can be reduced while ensuring service processing capabilities of the first device and a virtual machine running on the first device.

In another possible implementation, the processing module is further configured to determine the adjustment rate of the first QP based on the instant rate of the first QP, a weight coefficient corresponding to a priority of the first device, and the maximum sending rate of the first QP.

In another possible implementation, when determining the adjustment rate of the first QP, the processing module of the packet sending apparatus may further determine the adjustment rate of the first QP by considering a priority of the virtual machine on the first device. During specific implementation, a QP in the first device may establish a binding relationship with a specified virtual machine. In this case, all packets of the virtual machine are sent to a storage device by using the QP bound to the virtual machine. The processing module of the packet sending apparatus determines the adjustment rate of the first QP based on the instant rate of the first QP, the maximum sending rate of the first QP, and a service priority of a virtual machine in a first device.

In another possible implementation, when determining the adjustment rate of the first QP, the processing module of the packet sending apparatus may determine the adjustment rate of the first QP by simultaneously considering priorities of the first device and the virtual machine on the first device. When adjusting the rate at which the first QP sends a packet, the processing module of the packet sending apparatus determines the adjustment rate of the first QP not only by considering the instant rate of the first QP and the maximum sending rate of the first QP, but also by considering different service priorities of the first device and the virtual machine on the first device, thereby preferentially ensuring a processing process of a service having a high priority while relieving network congestion, and further improving reasonableness of congestion relief of a data transmission system.

In another possible implementation, each of at least one second device further establishes at least one QP between the packet sending apparatus and each of the at least one second device based on the RoCE protocol, and the first device and the second device include m QPs; and the processing module is configured to determine the adjustment rate S1of the first QP by using a formula

S1=D⁢A1/C1∑i=1m⁢Ai/Ci
and based on the instant rate A1of the first QP, the maximum sending rate C1of the first QP, and a port rate D during congestion of the switch, where Aiindicates an instant rate of an ithQP of the m QPs, Ciindicates a maximum sending rate of the ithQP of the m QPs, m is a positive integer, and i is a positive integer ranging from 1 to m.

A capability of forwarding a packet of the first QP by the switch during congestion of the switch can be determined by using the foregoing formula. In this way, the adjustment rate of the first QP is determined, and network congestion of the switch is relieved.

In another possible implementation, each of at least one second device further establishes at least one QP between the packet sending apparatus and each of the at least one second device based on the RoCE protocol, and the first device and the second device includes m QPs; and the processing module is configured to: determine the adjustment rate S1of the first QP by using a formula

S1=DB1⁢A1/C1∑i=1m⁢Ai/Ci
and based on the instant rate A1of the first QP, a weight coefficient B1corresponding to a priority of a first server, the maximum sending rate C1of the first QP, and a port rate D during congestion of the switch, where Aiindicates an instant rate of an ithQP of the m QPs, Ciindicates a maximum sending rate of the ithQP of the m QPs, m is a positive integer, and i is a positive integer ranging from 1 to m.

In another possible implementation, the obtaining module is configured to: receive the first packet that is from the first QP of the first device and that is forwarded by using the switch, read an ECN flag bit of the first packet, determine that the ECN flag bit of the first packet indicates that network congestion occurs on the switch, and obtain the instant rate of the first QP and the maximum sending rate of the first QP.

According to a third aspect, a storage device includes a processor, a memory, a communications interface, a network interface card, and a bus. The processor, the memory, the communications interface, and the network interface card are connected to each other and implement mutual communication by using the bus. The memory is configured to store a computer execution instruction. When a storage device runs, the processor executes the computer execution instruction in the memory, to perform the operation steps according to the first aspect or any possible implementation of the first aspect by using a hardware resource in the storage device.

According to a fourth aspect, a computer-readable storage medium stores an instruction. When run on a computer, the instruction causes the computer to execute an instruction of the method according to the first aspect or any possible implementation of the first aspect.

The implementations provided in the foregoing aspects may further be combined to provide more implementations.

