Applicants claim the foreign priority benefits under 35 U.S.C. xc2xa7 119 of European Patent Application No. 99113020.4, filed in the European Patent Office on Jul. 7, 1999. The European application is incorporated by reference into this application.
The present invention is from the area of notification systems and message brokering. In particular, the present invention relates to the area of processing notifications and in particular of publish and subscribe requests.
The present invention has a very general scope. Its basic principles can be applied in any situation in which any notification process or brokering, and in particular publish and subscribe processes take place.
The present invention introduces database technology into these matters of interest. It should be understood that for the sake of this invention the data model of the database system is irrelevant. Also, the database system might manage persistent data on disk or might manage data in main memory, i.e. a main memory database. Nevertheless, the terminology of relational database systems is persistently used in here for simplicity and increased clarity.
In general, a message broker is an instrument which is applied in a situation where messages are often and sometimes periodically sent from a plurality of message sources m to a plurality of message sinks n in a m/n relationship. Special cases of 1:m and n:1 occur as well. Message brokers xe2x80x98sharexe2x80x99 messages in a way that a sourcexe2x80x94often being any kind of program applicationxe2x80x94needs to transmit only one message and the broker delivers one or many versions of it to one or more sinks which are often applications as well. Further details on message broker systems can be found in R. Schulte, Message Brokers: A focused approach to application integration, Gartner Group, Strategic Analysis Report SSA R-401-102, 1996.
More particularly, a message broker is like a hub where messages are streaming in and are streaming out. The messages which are streaming into the broker are referred to hereinafter as being published. Messages which are streaming out of the broker are referred to as being subscribed. A subscription request specifies the subset of all incoming messages a particular application is interested in, and the format in which it has to be presented to the subscriber.
For example, various stock exchanges might publish stock data periodically, i.e. the stock data is sent to the message broker. Each stock exchange might use a different format to transfer its data. A subscriber might have registered to all messages about IBM stock data exceeding $103 and might request its delivery in XML format.
In state-of-the-art message brokers that provide publish/subscribe functionality significant effort is spent in managing the published messages and the subscriptions. This management is typically performed by using its own mechanisms. The present invention proposes to use object-relational database technology to implement publish/subscribe functionality.
State-of-the-art message brokers typically deal with messages that are being published and that subscribers would like to know about. However, subscribers are not just interested in messages that are being published, but also in changes that are applied to data stored in one or more tables in one or more databases. The present invention implements this functionality based on object-relational database technology.
Thus, it would be desirable to alleviate this big amount of programming, customizing effort. The above-cited prior art reference analyzes the feasibility of database technology used as a base for message brokering. It concludes that database systems lack many features that are necessary for message brokering, e.g., the middleware designed for performing the brokering, such as interface engines and message switches. Thus, a database technology as a base for message brokering is rejected in the above cited reference as being too difficult and complex to realize, and too slow in performance.
When analyzing the behavior of publish/subscribe technology using a database oriented view, the following observation can be made: a subscription request is similar to a query. Based on what has been specified in the subscription request, messages are filtered out and are delivered to its recipient. But there is a fundamental operational difference between a query and a subscription: A subscription does not operate on all messages stored for example in a message warehouse but only on a single message at a time, namely the message that has just been published to the message broker. Furthermore, typically many subscriptions will be registered with a message broker, i.e. identifying subscriptions with queries will result in the situation in which many queries will have to be evaluated on a single message: This situation reverses the situation typically found in query systems where the number of data items to be inspected exceeds the number of queries by many orders of magnitudes.
The proposal of the present invention to use object-relational database technology for publish/subscribe technology faces two problem areas that must be solved:
Firstly, how to efficiently identify operational data that is relevant to be published and secondly, how to perform subscriptions efficiently based on externalized database technology.
The first problem area results from the fact that most message brokers are not implemented based on database technology, and that in business environments having a natural need for management relevant enterprise data particular modifications of e.g., operational data are often seen as events that are relevant for external applications. This is depicted in FIG. 1.
For example, if a new tuple is inserted into a table, a person might want to be informed about this fact via a corresponding e-mail, or an application must be invoked because of this, or a message broker in turn has subscribers that want to get this data. Thus, the database may be seen as a source for publications.
In particular, when a message broker is used to manage the subscription requests for the changes in operational data and thus the operational data changes need to be published to the message broker, the importance of pre-filtering of publications becomes obvious. If each of the huge number of changes of operational data is pushed to the message broker which then determines whether or not a subscriber for the subject change exists, lots of data may be processed unnecessarily. This might be a waste of resources. Thus, providing subscription functionality within a database system is beneficial, in general.
Otherwise, without having subscription features natively provided within the database system the modified data has to be transformed into a message, then has to be sent, i.e. published to a separate message broker, and this message broker has to determine all corresponding subscribers. In this approach the database would be considered to be a publisher only, the message broker""s subscription engine is used to determine whether or not any subscriber is interested in the modified data.
If no subscriber is interested in this data communication between the database system and the message broker including associated data transformations etc. then much work and traffic is performed in vain putting unnecessary load onto the overall environment. Furthermore, a usage of a separate subscription engine would ignore the fact, that the database system""s query capability can be immediately used to process the subscriptions directly without needing a separate message broker enginexe2x80x94this will further improve efficiency.
The second problem deals with efficiency of implementing subscriptions based on database technology itself. Nearly all relational database systems are supporting trigger mechanisms today. Triggers allow the database system to automatically perform an action based on a registered interest on relevant modifications of data. There seems to be a suggestion for recognizing a xe2x80x98naturalxe2x80x99 fit between triggers and subscriptions. Simply map each subscription request onto a corresponding trigger.
For example, assuming that a STOCK table contains rows reporting the NAME of a company, its actual stock PRICE, and the AMOUNT traded. Then, each subscription on modified stock data is mapped to a separate trigger in this table. Please refer to FIG. 2 for a sample mapping of a particular subscription. If a number of S(T) subscribers, with T as an abbreviation for xe2x80x98tablexe2x80x99, have registered for modifications on a given table T, and a number of n(T) modifications happen per second on this table, then S(T)xc3x97n(T) triggers have to be run per second for table T. In practice, this number is even larger because more than one table will be of interest for subscriptions. If there are 10 modifications on the stock table per second and 100 subscriptions, both moderate estimates, 1,000 triggers have to be run on this single table. This is out of the scope of current database technology.
Therefore, the object of the present invention is to provide a method and system for efficient usage of database technology for notifications, and in particular for subscriptions.
These objects of the invention are achieved by the features stated in enclosed independent claims. Further advantageous arrangements and embodiments of the invention are set forth in the respective subclaims.
The present invention teaches in general how to efficiently exploit active database technology and extensible database technology, i.e. triggers and user-defined functions, for processing subscriptions.
According to a first aspect of the present invention it is proposed to aggregate all subscriptions on a certain table, or more generally of a plurality of tables, into a single trigger on this table or tables, respectively. This will dramatically improve the trigger-based efficiency.
According to a second aspect of the present invention it is proposed to deliver the information a subscriber has registered for, by exploiting the database system provided facility of user-defined functions. This allows the creation of the information to be sent to the subscriber(s) from the address space the database system is running, which is another source of efficiency improvement.
The overall processing can thus be summarized as follows:
When a first subscription is registered for a table T this kind of xe2x80x9caggregated subscription triggerxe2x80x9d is created on the table T, and a collection of so-called metadata tables suitable to hold all subscriptions for the table T is created which is called ENABLE TABLE, depicted in FIG. 3. It should be noted that this function can also be realized as a separate utility explicitly to be invoked by an administrator, for example.
Preferred embodiments for creating suitable metadata tables and for creating aggregated subscription triggers are described later in more detail.
Each single subscription is reflected by xe2x80x9csimplyxe2x80x9d INSERTing appropriate tuples into these latter metadata tables, i.e. according to the invention no separate trigger is created and no modification on the single subscription trigger has to be applied.
A preferred embodiment for reflecting a subscription as tuples in the associated metadata tables is described in detail.
When a tuple is manipulated in table T which corresponds to a publication in a usual prior art message broker the corresponding trigger fires and determines in a single invocation all subscribers interested in modification of tuples.
Using a single subscription trigger to achieve the desired efficient usage of the database system""s trigger features has its price in that. The query in the body of the trigger must be a more complicated one as it basically has to search for all potential subscribers in the associated collected metadata tables. Depending on the embodiment of the present invention this might involve at least an n-way join where n is the number of comparison operators supported in subscription filters for the target table. Nevertheless, query systems have been optimized for decades to perform n-way joins very efficiently.
With reference to the notification which is initiated by the database trigger it should be mentioned that the inventional concept is very general in this respect. Notification could mean, for example, that a message in the sense of a messaging system is sent, an action is performed on an object managed on an object server, or, an e-mail is sent to the subscriber. It could be even inserted into a table that is maintained for each subscriber, which allows the subscriber to retrieve the appropriate notifications by querying the table.
With respect to the second aspect of the present invention as mentioned above the actual construction of the notification to be sent to each qualified subscriber can be done based on a so-called user-defined function (UDF). While the filter of the query in the body of the trigger determines the qualifying subscribers, the SELECT-clause of the query may reference a UDF which initiates the delivery of the notification to each subscriber. This delivery UDF is automatically invoked by the database system for each qualified subscriber. The delivery function will get all data necessary, e.g., the relevant values of the modified tuple which caused the firing of the trigger, the recipients address, etc., to construct the notification. Based on this data passed, the delivery function will construct the notification and send it to the recipient""s address, e.g. via a message queuing system like MQSeries, via e-mail etc. A preferred embodiment of the delivery UDF is described in more detail later.
As an advantage, the present invention builds an efficient subscription mechanism on top of existing database functionality. No modifications to the database system are required. Thus, a standard object-relational database system can be utilized, as it is, to implement a publish/subscribe engine. Furthermore, the subscription mechanism described easily transforms a database into a publisher of messages.