BACKGROUND

In traditional database systems, data is stored in one or more databases usually in the form of tables. The stored data is then queried and manipulated using a data management language such as a structured query language (SQL). For example, a SQL query may be defined and executed to identify relevant data from the data stored in the database. A SQL query is thus executed on a finite set of data stored in the database. Further, when a SQL query is executed, it is executed once on the finite data set and produces a finite static result. Databases are thus best equipped to run queries over finite stored data sets.

A number of modern applications and systems however generate data in the form of continuous data or event streams instead of a finite data set. Examples of such applications include but are not limited to sensor data applications, financial tickers, network performance measuring tools (e.g. network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, and the like. Such applications have given rise to a need for a new breed of applications that can process the data streams. For example, a temperature sensor may be configured to send out temperature readings.

Managing and processing data for these types of event stream-based applications involves building data management and querying capabilities with a strong temporal focus. A different kind of querying mechanism is needed that comprises long-running queries over continuous unbounded sets of data. While some vendors now offer product suites geared towards event streams processing, these product offerings still lack the processing flexibility required for handling today's events processing needs.

BRIEF SUMMARY

Techniques are provided (e.g., a method, a system, non-transitory computer-readable medium storing code or instructions executable by one or more processors) for processing events of a continuous event stream. In an embodiment, a distributed event processing system is disclosed that can receive one or more continuous data streams, register a continuous query against the data streams, and continuously execute the query as new data appears in the streams. In certain embodiments, the distributed event processing system may be configured to deploy and execute applications (e.g., event processing applications) by distributing the execution of the application on a cluster of machines within the system. In certain embodiments, the distributed event processing system may be configured to receive updates to an application, process the updated application against a continuous stream of data, and continuously generate updated results to a user. In certain embodiments, the system reduces delays associated with application deployment and execution when updates to an application are received and enables a user to view results pertaining to an updated application quickly without experiencing the typical deployment and execution delays that occur when updates to an application are made.

In certain embodiments, the distributed event processing system comprises a memory storing a plurality of instructions and one or more processors configured to access the memory. The processors are configured to receive information that identifies an application, determine that the application is to be deployed, and transmit an instruction to deploy the application to a cluster of computing nodes in the distributed event processing system. The processors are further configured to receive, at a first service, information that identifies a location of a computing node in the cluster of computing nodes where the application is deployed and store the information that identifies the location of the computing node.

In certain embodiments, the processors are configured to receive event data related to the application via an event stream, transmit the event data to the computing node in the cluster of computing nodes for processing, receive results related to processing of the event data from the computing node in the cluster of computing nodes and transmit the results to a user of the first computing device.

In certain examples, the processors are configured to receive an update to the application, determine that the application is deployed on the computing node in the cluster of computing nodes, and transmit the update to a second service on the cluster of computing nodes. In certain examples, the processors are configured to determine that the application is deployed on the computing node based on the information that identifies the location of the computing node, stored in the computing node. In an example, the information that identifies the location of the computing node in the cluster of computing nodes is determined by the second service associated with the cluster of computing nodes.

In certain examples, the computing node in the cluster of computing nodes is configured to receive, from the second service, the update to the application, identify a portion of a Directed Acyclic Graph (DAG) of the application that corresponds to the update, and generate an updated DAG for the application based on the identified portion. In certain examples, the processors are configured to receive the event data related to the application, transmit the event data to the computing node in the cluster of computing nodes for processing, and receive updated results related to the processing of the event data relative to the updated DAG of the application, from the computing node in the cluster of computing nodes. In certain examples, the processors are configured to provide updated results to the user.

In some examples, the first service is a Representational State Transfer (REST) service associated with the first computing device, the second service is a REST service associated with the cluster of computing nodes, and the information that identifies the location of the computing node is a Universal Resource Locator (URL) of the computing node on the cluster of computing nodes.

In some examples, the computing node in the cluster of computing nodes is configured to generate a DAG of transformations associated with the application, identify a set of tasks to be executed based on the DAG of transformations, invoke a set of worker computing nodes to perform the set of tasks, where the set of tasks are identified based on the set of one or more transformations associated with the application, and execute the set of tasks relative to the event data received via the event stream. In some examples, the computing node in the cluster of computing nodes is configured to generate results related to processing of the event data based on the execution of the set of tasks and transmit the results to the user.

The techniques described above and below may be implemented in a number of ways and in a number of contexts. Several example implementations and contexts are provided with reference to the following figures, as described below in more detail. However, the following implementations and contexts are but a few of many.

