CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Patent Application Ser. No. 62,854,050 filed May 29, 2019 which is incorporated herein by reference in its entirety.

FIELD OF THE DISCLOSURE

This disclosure relates to monitoring network packets within network communication systems and, more particularly, to monitoring communication systems to detect dropped aggregated traffic metadata packets

BACKGROUND

It is useful for operators of communication networks to be alerted when aggregated traffic metadata packets (e.g., NetFlow packets or the equivalent) aggregated by a traffic management device (e.g., a router) and sent to a collecting device (hereafter called a collector) are dropped. Aggregated traffic metadata packets may be dropped for numerous reasons, such as network congestion or network outages. When aggregated traffic metadata packets are dropped due to network congestion, it is commonly referred to as ‘dropped flow’ or ‘flow is being dropped’. For communication network systems that process aggregated traffic metadata packets, dropped flow can be detected by tracking monotonically increasing sequence numbers that appear in each aggregated traffic metadata packet. These sequence numbers can be inserted by one or more aggregation engines included in each traffic management device.

Determining if aggregated traffic metadata packets are being dropped is often complex. Aggregated traffic metadata packets are not guaranteed to arrive at a collector in the order that they were transmitted. A jump greater than one in sequence numbers of aggregated traffic metadata packets output by a given aggregation engine of a traffic management device (referred to as an origination pair) can be due to tardy aggregated traffic metadata packets, and therefore does not necessarily imply that an aggregated traffic metadata packet has been dropped.

In fact, in some methods of aggregation used by traffic management devices, aggregating traffic metadata packets can contain more than one record of metadata, with the sequence number of each aggregated traffic metadata packet being incremented by the number of records contained in the previous aggregated traffic metadata packet. In an effort to parallelize work over different CPU cores, complexity can increase when aggregated traffic metadata packets from a single origination pair are distributed to several cooperating collector processes for processing. A single collector process is likely to not see all the aggregated traffic metadata packets from a given origination pair. For instance, a collector process may become so overloaded that it is unable to properly process aggregated traffic metadata packets or summary messages summarizing aggregated traffic metadata packets. When this overload condition occurs, aggregated traffic metadata packets or the corresponding summary messages can be dropped, even after the aggregated traffic metadata packets have arrived at the collector. Although aggregated traffic metadata packets or their summary messages that are dropped after the aggregated traffic metadata packets have been accepted by the collector are not considered dropped aggregated traffic metadata packets, it is very useful to know when such an event has occurred.

SUMMARY

The purpose and advantages of the below described illustrated embodiments will be set forth in and apparent from the description that follows. Additional advantages of the illustrated embodiments will be realized and attained by the devices, systems and methods particularly pointed out in the written description and claims hereof, as well as from the appended drawings.

To achieve these and other advantages and in accordance with the purpose of the illustrated embodiments, in one aspect, a method is provided of determining dropped aggregated traffic metadata packets that were output from at least one traffic management device to at least one collector device. The method includes receiving summary messages summarizing respective aggregated traffic metadata packets output from the at least one traffic management device and an engine. Each traffic management device includes at least one engine. Each summary message corresponds to one of the aggregated traffic metadata packets and identifies an origination pair and a sequence number assigned by the origination pair. The origination pair includes a traffic management device of the at least one traffic management device and an engine of the traffic management device from which the aggregated traffic metadata packet was output. The method further includes tracking per subinterval of a series of sub-intervals, highest and lowest sequence numbers and a count of summary messages received for each unique origination pair from the beginning of the subinterval, wherein each series of subintervals is included within a respective interval of a series of intervals. The method further includes accumulating, per interval, accumulated highest and lowest sequence numbers and an accumulated count of summary messages for each unique origination pair from the beginning of the interval and for all previous subintervals of the series of subintervals included in the interval for tracking dropped aggregated traffic metadata packets for the interval. The method further includes estimating a number of dropped aggregated traffic metadata packets for the interval based on the accumulated highest and lowest sequence numbers and the count of summary messages.

In accordance with one or more embodiments, the method can further include receiving a data lost notification for one of the subintervals, the data lost notification indicating a data lost event in which information regarding one or more aggregated traffic metadata packets was knowingly discarded from one of the received summary messages due to a full condition of a channel by which the summary messages are received. The method can further include adjusting the tracked highest and lowest sequence numbers and a count of summary messages for the subinterval to indicate that the aggregated traffic metadata packets were not dropped during the subinterval.

In accordance with one or more embodiments, the method can further include changing states of a state machine to handle and recover from receiving a data lost indication.

In accordance with one or more embodiments, tracking during a current subinterval of a series of subintervals can include attributing a received aggregated traffic metadata packet to either the current subinterval or a previous subinterval of the series of subintervals that precedes the current subinterval based on a comparison of the sequence number of the aggregated traffic metadata packet to the highest sequence number of any of the previous subintervals of the series of subintervals.

In accordance with one or more embodiments, accumulating during a current interval of the series of intervals for a unique origination pair can include updating the accumulated highest and lowest sequence numbers and the accumulated count of summary messages using the highest and lowest sequence numbers and a count of summary messages at the end of each subinterval of the series of subintervals included in the current interval that is not a first subinterval of the series of subintervals included in the current interval.

In accordance with one or more embodiments, accumulating during a current interval of the series of intervals for a unique origination pair can include clearing the accumulated highest and lowest sequence numbers and the accumulated count of summary messages at the end of a first subinterval of the series of subintervals included in the current interval.

In accordance with one or more embodiments, estimating the number of dropped aggregated traffic metadata packets for the interval can include setting, for a unique origination pair, an expected summary messages value to equal a difference between the accumulated highest and lowest sequence numbers incremented by one of the unique orientation pair, and totaling, for the unique origination pair, for the subintervals included in the interval, a difference between the expected summary messages value and the accumulated count of summary messages.

In accordance with one or more embodiments, the method can further include processing the summary messages received, outputting telemetry data about the aggregated traffic metadata packets based on information determined by processing the summary messages, and adjusting an action applied for managing or monitoring network traffic based on the telemetry data.

In accordance with aspects of the disclosure, a system is provided to perform the method of the disclosure. Furthermore, in accordance with aspects of the disclosure a non-transitory computer readable storage medium and one or more computer programs stored therein, the computer programs comprising instructions, which when executed by a computer system, cause the computer system to perform the method of the disclosure.

