The present application claims priority to Chinese patent application No. 201910543654.1, filed with the China National Intellectual Property Administration on Jun. 21, 2019 and entitled “Network Function Virtualization Intelligent Scheduling Method Based on Traffic Identification”, which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present invention relates to communication technologies, and more particular, to a network resource scheduling method, apparatus, an electronic device and a storage medium.

BACKGROUND

As network traffic has increased and network applications have been widely used, users of the network are requiring a higher quality of service. The quality of service requirements may include low transmission latency, low packet loss rate, high throughput, etc. To satisfy the requirements of the users, not only an increase in bandwidth and capacity but also an appropriate resource scheduling mechanism has been desirable for network devices.

Network Function Virtualization (NFV) is a technique that may separate network functions from traditional hardware devices, and can implement network functions in a flexible way arranged and share network resources. NFV may use Virtualized Network Functions (VNF) to create, configure, monitor and operate different networks. In a content distribution network that includes multiple network nodes, VNFs are deployed in different network nodes, and network services are realized by a service function chain composed of certain VNFs. Therefore, when the NFV technique is used, VNFs are arranged and scheduled to provide a service function chain for network services, which can improve the quality of services of the network.

However, existing scheduling solutions follow the first-in first-out principle, with a goal of improving network utilization, to complete transmission of as much network traffic as possible. That is, data streams are processed in the order in which the data streams are received. The data stream received first will be handled first, and the data stream received next will handled later. However, different types of data streams require different quality of services. In the first-in first-out solutions, the data stream with a lower priority may occupy a large amount of network resources, which may result in a high delay of the data stream having a higher priority and a poor performance of network services.

SUMMARY

The objective of embodiments of the present application is to provide a network resource scheduling method, apparatus, an electronic device and a storage medium, to improve the performance of network services.

In a first aspect, an embodiment of the present application provides a network resource scheduling method. The method is applicable to an ingress node in a network cluster, and includes:

upon receipt of a network data stream, determining a traffic type of the network data stream based on the number of data packets of the network data stream received within a specified period of time, lengths of the data packets and reception times of the data packets;

for each data packet included in the network data stream, determining a target transmission path for the data packet, based on node state parameters of nodes in the network cluster, link state parameters of links in the network cluster, and the traffic type of the network data stream when the data packet is received; and

transmitting the data packet via the target transmission path.

In an embodiment, the determining of a target transmission path for the data packet includes:

when the data packet is received, collecting the node state parameters of the nodes in the network cluster, wherein a node state parameter of a node represents an idle computing resource of the node;

collecting the link state parameters of the links in the network cluster, wherein a link state parameter of a link represents an idle bandwidth resource of the link;

determining transmission state parameters of the data packet based on the traffic type of the network data stream, a preset correspondence between traffic types and maximum transmission times, and a preset correspondence between the traffic types and transmission priorities; wherein the transmission state parameters include a maximum transmission time and a transmission priority of the data packet; and

inputting the node state parameters, the link state parameters and the transmission state parameters into a network scheduling model to obtain the target transmission path output from the network scheduling model.

In an embodiment, the network scheduling model is obtained through operations including:

setting a state space for a Trust Region Policy Optimization (TRPO) algorithm:
Sti={SNti,SEti,SPti}={C1ti, . . . ,Cnti,b1ti, . . . ,bnti,MTTi,TPi};

wherein Stiis a network state at a time step ti, and Stiis composed of SNti, SEtiand SPti; and the time step tiis a time step at which the network scheduling model performs network resource scheduling for a data packet i;

wherein SNtiis the node state parameters of the nodes in the network cluster, and SNtiincludes C1ti, . . . , CNti, where CNtirepresents an idle computing resource of a node n at the time step ti;

wherein SEtiis the link state parameters of the links in the network cluster, and SEtiincludes b1ti, . . . , bNti, where bNtirepresents an idle bandwidth resource of a link n at the time step ti; and

wherein SPtiincludes MTTiand TPi; MTTirepresents a maximum transmission time of the data packet i, and the maximum transmission time of the data packet i is a maximum transmission time corresponding to the traffic type of the network data stream to which the data packet i belongs; and TPirepresents a transmission priority of the data packet i, and the transmission priority of the data packet i is a transmission priority corresponding to the traffic type of the network data stream to which the data packet i belongs;

traversing the network cluster to determine transmission paths from the ingress node to an egress node in the network cluster, and setting an action space for the TRPO algorithm based on the determined transmission paths: A={a|a∈{1, 2, . . . , P}}, wherein a represents an action selected according to the TRPO algorithm, and P represents the number of the transmission paths from the ingress node to the egress node in the network cluster;

setting a reward function for the TRPO algorithm:

rt=-∑i∈It1MTTi-cNt+b;

wherein rtis the reward function at the time step t, Itis a set of data packets transmitted over the network cluster within the time step t, Ntis the number of data packets discarded within the time step t, b is a baseline value, and c is a penalty coefficient;

setting a loss function for the TRPO algorithm:

L⁡(θ)=Et[clip(rt(θ),1-ε,1+ε)⁢A^t-λ⁢Dpp(πθold(·|s),πθ(·|s))];

wherein

Et[clip(rt(θ),1-ε,1+ε)⁢A^t-λ⁢Dpp(πθold(·|s),πθ(·|s))]
represents a mathematical expectation of

clip(rt(θ),1-ε,1+ε)⁢A^t-λ⁢Dpp(πθold(·|s),πθ(·|s))
at the time step t, θ represents a set of all parameters of a neural network for a policy in the TRPO algorithm, and

rt(θ)=πθ(at|st)πθold(at|st)
represents a ratio of a probability that an action is selected under a new policy in a network state to a probability that the same action is selected under an old policy in the same network state;

wherein the function clip( ) is a value clipping function; 1+ε(ε∈(0,1)) in the function clip( ) is an upper bound value of rt(θ), and 1−ε(ε∈(0,1)) is a lower bound value of rt(θ); if a calculated value of rt(θ) is between the upper bound value and the lower bound value, a value of the function clip( ) is the calculated value of rt(θ); if the calculated value of rt(θ) is greater than the upper bound value, the value of the function clip( ) is the upper bound value; and if the calculated value of rt(θ) is less than the lower bound value, the value of the function clip( ) is the lower bound value;

wherein

Dpp(πθold(·|s),πθ(·|s))=(πθold(at|st)-πθ(at|st))2
represents a square value of a distance of a point probability distribution between the new policy πθ(at|st) and the old policy

πθold(at|st),
and λ is a penalty coefficient;

wherein Âtis an advantage function, Âtrepresents a difference between an action value function for selecting a specific action at the time step t and a state value function at the time step t,

Aˆt=A⁡(s,a)=Eπ[∑k=0∞γt⁢rt+k|St=s,At=a]-Vt(s);

wherein

Eπ[∑k=0∞γt⁢rt+k|St=s,At=a]
represents the action value function for taking an action a under a states according to a policy

π,∑k=0∞γt⁢rt+k
is a predicted cumulated reward value in K time steps after taking the action a at the time step t, Eπ[ ] is used to compute an expectation; γ(γ∈(0,1)) is a discount factor indicating importance of future rewards relative to current rewards; rt+krepresents a reward value at a time step t+k; Vt(s) represents a weighted sum of action value functions of all actions in the action space under the network state s at the time step t; and

training the neural network model by using network data stream samples, the state space, the action space, the reward function and the loss function to obtain the network scheduling model, wherein the neural network model is a neural network model based on the TRPO algorithm.

In an embodiment, the determining of a traffic type of the network data stream includes:

determining features of the network data stream based on the number of the data packets of the network data stream received within the specified period of time, the lengths of the data packets and the reception times of the data packets; and

inputting the features into a preset classification model to obtain a result of classification of the network data stream from the preset classification model, wherein the preset classification model is obtained by training with a preset training set, wherein the preset training set includes network data stream samples and traffic types of the network data stream samples;

wherein the features include:

within the specified period of time, a maximum value of arrival time intervals of received forward packets of the network data stream, a minimum value of the arrival time intervals of the forward packets, an average value of the arrival time intervals of the forward packets, a standard deviation of the arrival time intervals of the forward packets, a maximum value of lengths of the forward packets, a minimum value of the lengths of the forward packets, an average value of the lengths of the forward packets, and a standard deviation of the lengths of the forward packets;

within the specified period of time, a maximum value of arrival time intervals of received backward packets of the network data stream, a minimum value of the arrival time intervals of the backward packets, an average value of the arrival time intervals of the backward packets, a standard deviation of the arrival time intervals of the backward packets, a maximum value of lengths of the backward packets, a minimum value of the lengths of the backward packets, an average value of the lengths of the backward packets, and a standard deviation of the lengths of the backward packets;

within the specified period of time, a maximum value of arrival time intervals of received bidirectional packets of the network data stream, a minimum value of the arrival time intervals of the bidirectional packets, an average value of the arrival time intervals of the bidirectional packets, a standard deviation of the arrival time intervals of the bidirectional packets, a maximum value of lengths of the bidirectional packets, a minimum value of the lengths of the bidirectional packets, an average value of the lengths of the bidirectional packets, and a standard deviation of the lengths of the bidirectional packets;

duration of the network data stream within the specified period of time, wherein the duration of the network data stream refers to a difference between a reception time of a first data packet and a reception time of a last data packet within the specified period of time; and

within the specified period of time, the number of the received forward packets of the network data stream per second, the number of bytes of the forward packets per second, the number of the received backward packets of the network data stream per second, the number of bytes of the backward packets per second, a ratio of the number of the forward packets per second to the number of the backward packets per second, and a ratio of the number of the bytes of the forward packets per second to the number of the bytes of the backward packets per second.

In an embodiment, after determining the traffic type of the network data stream, the method further includes:

re-determining, every specified time interval, the traffic type of the network data stream based on the number of data packets of the network data stream received within the specified period of time, the lengths of the data packets and the reception times of the data packets.

In a second aspect, an embodiment of the present application provides a network resource scheduling apparatus. The apparatus is applicable to the ingress node in a network cluster, and includes:

a determination module, configured for upon receipt of a network data stream, determining a traffic type of the network data stream based on the number of data packets of the network data stream received within a specified period of time, lengths of the data packets and reception times of the data packets; and for each data packet included in the network data stream, determining a target transmission path for the data packet, based on node state parameters of nodes in the network cluster, link state parameters of links in the network cluster, and the traffic type of the network data stream when the data packet is received; and

a transmission module, configured for transmitting the data packet via the target transmission path.

In an embodiment, the determination module is further configured for:

when the data packet is received, collecting the node state parameters of the nodes in the network cluster, wherein a node state parameter of a node represents idle computing resources of the node;

collecting the link state parameters of the links in the network cluster, wherein a link state parameter of a link represents idle bandwidth resources of the link;

determining transmission state parameters of the data packet based on the traffic type of the network data stream, a preset correspondence between traffic types and maximum transmission times, and a preset correspondence between the traffic types and transmission priorities; wherein the transmission state parameters include a maximum transmission time and a transmission priority of the data packet; and

inputting the node state parameters, the link state parameters and the transmission state parameters into a network scheduling model to obtain the target transmission path output from the network scheduling model.

In an embodiment, the apparatus further includes:

a training module, configured for training the network scheduling model through operations including:

setting a state space for a Trust Region Policy Optimization (TRPO) algorithm:
Sti={SNti,SEti,SPti}={C1ti, . . . ,Cnti,b1ti, . . . ,bnti,MTTi,TPi};

wherein Stiis a network state at a time step ti, and Stiis composed of SNti, SEtiand SPti; and the time step tiis a time step at which the network scheduling model performs network resource scheduling for a data packet i;

wherein SNtiis the node state parameters of the nodes included in the network cluster, and SNtiincludes C1ti, . . . , CPti, where Cntirepresents an idle computing resource of a node n at the time step ti;

wherein SEtiis the link state parameters of the links included in the network cluster, and SEtiincludes b1ti, . . . , bnti, where bntirepresents an idle computing resource of a link n at the time step ti; and

wherein SPtiincludes MTTiand TPi; MTTirepresents the maximum transmission time of the data packet i, and the maximum transmission time of the data packet i is a maximum transmission time corresponding to the traffic type of the network data stream to which the data packet i belongs; and TPirepresents the transmission priority of the data packet i, and the transmission priority of the data packet i is a transmission priority corresponding to the traffic type of the network data stream to which the data packet i belongs;

traversing the network cluster to determine transmission paths from the ingress node to an egress node in the network cluster, and setting an action space for the TRPO algorithm based on the determined transmission paths: A={a|a∈{1, 2, . . . , P}}, wherein a represents an action selected according to the TRPO algorithm, and P represents the number of the transmission paths from the ingress node to the egress node in the network cluster;

setting a reward function for the TRPO algorithm:

rt=-∑i∈It1MTTt-cNt+b;

wherein rtis the reward function at the time step t, Itis a set of data packets transmitted over the network cluster within the time step t, Ntis the number of data packets discarded within the time step t, b is a baseline value, and c is a penalty coefficient;

setting a loss function for the TRPO algorithm:

L⁡(θ)=Et[clip(rt(θ),1-ε,1+ε)⁢A^t-λ⁢Dpp(πθold(·|s),πθ(·|s))];

wherein

Et[clip(rt(θ),1-ε,1+ε)⁢A^t-λ⁢Dpp(πθold(·|s),πθ(·|s))]
represents a mathematical expectation of

clip(rt(θ),1-ε,1+ε)⁢A^t-λ⁢Dpp(πθold(·|s),πθ(·|s))
at the time step t, θ represents a set of all parameters of a neural network for the policy in the TRPO algorithm, and

rt(θ)=πθ(at|st)πθold(at|st)
represents a ratio of a probability that an action is selected under a new policy πθ(at|st) in a network state to a probability that the same action is selected under an old policy

πθold(at|st)
in the same network state;

wherein the function clip( ) is a value clipping function; 1+ε(ε∈(0,1)) in the function clip( ) is an upper bound value of rt(θ), and 1−ε(ε∈(0,1)) is a lower bound value of rt(θ); if a calculated value of rt(θ) is between the upper bound value and the lower bound value, a value of the function clip( ) is the calculated value of rt(θ); if the calculated value of rt(θ) is greater than the upper bound value, the value of the function clip( ) is the upper bound value; and if the calculated value of rt(θ) is less than the lower bound value, the value of the function clip( ) is the lower bound value;

wherein

Dpp(πθdd(·|s),πθ(·|s))=(πθold(at|st)-πθ(at|st))2
represents a square value of a distance of a point probability distribution between the new policy πθ(at|st) and the old policy

πθold(at|st),
and λ is the penalty coefficient;

wherein Âtis an advantage function, Âtrepresents a difference between an action value function for selecting a specific action at the time step t and the state value function at the time step t,

A^t=A⁡(s,a)=Eπ[∑k=0∞γt⁢rt+k|St=s,At=a]-Vt(s);

wherein

Eπ[∑k=0∞γt⁢rt+k|St=s,At=a]
represents the action function for taking an action a under a state s according to a policy

π,∑k=0∞γt⁢rt+k
is the predicted cumulated reward value in K time steps after taking action a at time step t, Eπ[ ] represents to computed the expectation; γ(γ∈(0,1)) is a discount factor to indicating importance of future rewards relative to current rewards; rt+krepresents the reward value at a time step t+k; Vt(s) represents the weighted sum of the action value functions of all the actions in the action space under the network state s at the time step t; and

training the neural network model by using network data stream, the state space, the action space, the reward function and the loss function to obtain the network scheduling model, wherein the neural network model is a neural network model based on the TRPO algorithm.

In an embodiment, the determination module is further configured for:

determining features of the network data stream based on the number of the data packets of the network data stream received within the specified period of time, the lengths of the data packets and the reception times of the data packets; and

inputting the features into a preset classification model to obtain a result of classification of the network data stream from the preset classification model, wherein the preset classification model is obtained by training with a preset training set, wherein the preset training set includes network data stream and traffic types of the network data stream as labels;

wherein the features include:

within the specified period of time, a maximum value of arrival time intervals of received forward packets of the network data stream, a minimum value of the arrival time intervals of the forward packets, an average value of the arrival time intervals of the forward packets, a standard deviation of the arrival time intervals of the forward packets, a maximum value of lengths of the forward packets, a minimum value of the lengths of the forward packets, an average value of the lengths of the forward packets, and a standard deviation of the lengths of the forward packets;

within the specified period of time, a maximum value of arrival time intervals of received backward packets of the network data stream, a minimum value of the arrival time intervals of the backward packets, an average value of the arrival time intervals of the backward packets, a standard deviation of the arrival time intervals of the backward packets, a maximum value of lengths of the backward packets, a minimum value of the lengths of the backward packets, an average value of the lengths of the backward packets, and a standard deviation of the lengths of the backward packets;

within the specified period of time, a maximum value of arrival time intervals of received bidirectional packets of the network data stream, a minimum value of the arrival time intervals of the bidirectional packets, an average value of the arrival time intervals of the bidirectional packets, a standard deviation of the arrival time intervals of the bidirectional packets, a maximum value of lengths of the bidirectional packets, a minimum value of the lengths of the bidirectional packets, an average value of the lengths of the bidirectional packets, and a standard deviation of the lengths of the bidirectional packets;

duration of the network data stream within the specified period of time, wherein the duration of the network data stream refers to a difference between a reception time of a first data packet and a reception time of a last data packet within the specified period of time; and

within the specified period of time, the number of the received forward packets of the network data stream per second, the number of bytes of the forward packets per second, the number of the received backward packets of the network data stream per second, the number of bytes of the backward packets per second, a ratio of the number of the forward packets per second to the number of the backward packets per second, and a ratio of the number of the bytes of the forward packets per second to the number of the bytes of the backward packets per second.

In an embodiment, the determination module is further configured for: re-determining, every specified time interval, the traffic type of the network data stream based on the number of data packets of the network data stream received within the specified period of time, the lengths of the data packets and the reception times of the data packets.

In a third aspect, an embodiment of the present application provides an electronic device, including:

a processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

the memory is configured for storing a computer program; and

the processor is configured for, by executing the computer program stored on the memory, implementing the network resource scheduling method provided by the embodiments of the present application.

In a fourth aspect, an embodiment of the present application provides a non-transitory computer readable storage medium having stored thereon a computer program that, when executed by a processor, cause the processor to carry out the network resource scheduling method provided by the embodiments of the present application.

In a fifth aspect, an embodiment of the present application provides a computer program product including instructions. The computer program product including the instructions, when executed on a computer, causes the computer to perform the network resource scheduling method provided by the embodiments of the present application.

The advantages of the embodiments of the present application include the following aspects.

In the solutions described herein, when a network data stream is received, the traffic type of the network data stream can be determined based on the number of, the lengths of and the reception times of data packets of the network data stream received within a specified period of time. Thus, no matter whether the network data stream is transmitted in an encrypted manner or not, the traffic type of the network data stream can be determined. Further, for each received data packet included in the network data stream, a target transmission path for the data packet can be determined based on the node state parameters of the nodes in the network cluster, the link state parameters of the links in the network cluster, and the traffic type of the network data stream. The data packet is then transmitted via the target transmission path. It can be seen that the embodiment of the present application also combines the traffic type of the network data stream when determining the transmission path for the data packet, so that the selected target transmission path meets the transmission requirements of the traffic type, improving the network service performance.

Of course, embodiments of the products or methods are not necessarily required to achieve all of the advantages described above at the same time.

