TECHNICAL FIELD

An embodiment of the present subject matter relates generally to message routing and, more specifically, to message routing optimization.

BACKGROUND

Routing providers provide message delivery functionality to customers. For example, routing providers charge customers a fee for each message delivered by the routing provider to an intended recipient by the routing provider. Customers may select which routing provider to use based on the performance provided by the routing providers. For example, customers may select a routing provider based on how likely it is that a message transmitted by the routing provider will be successfully delivered to its intended recipient.

Currently the process of selecting a routing provider is performed manually. For example, a human reviewer analyzes data indicating performance of the available routing providers and adjust use of the routing providers to optimize performance. This manual process is both resource intensive and slow as relies on human reviewers to both monitor the data and make adjustments. Accordingly, improvements are needed.

SUMMARY

A message routing optimization system monitors performance of multiple routing providers and allocates messages to the routing providers to optimize message routing performance. For example, the message routing optimization system allocates messages to the routing providers based on a conversion rate index that includes individual conversion rates determined for each routing provider. Each conversion rate indicates a performance level of a routing provider, such as by indicating an estimated likelihood that a message allocated to the routing provider will be successfully delivered to its intended recipient. The message routing optimization system allocates messages to the routing providers based on the conversion rate index, such as by allocating messages to the routing provider with the highest conversion rate. Messages are therefore allocated to the routing provider that is determined to provide the highest level or performance, thereby optimizing message routing performance.

The message routing optimization system calculates each conversion rate based on feedback data for each routing provider. The feedback data may include both live feedback data and testing feedback data. Live feedback describes each routing provider's performance delivering messages that have been allocated to the routing provider by the message routing optimization system. For example, the live feedback data can include data describing whether the routing provider transmitted the allocated message to its intended recipient, whether the message was successfully received by its intended recipient, whether the message resulted in a specified action by the recipient, an amount of time that elapsed until the message was transmitted by the routing provider, an amount of time that elapsed until the message was received by the recipient, and the like.

The testing feedback data describes the tested performance of the routing providers. For example, tests of a routing provider's network can be conducted during which test messages and/or synthetic messages are transmitted using routes provided by the routing provider. The testing feedback data describes the tracked performance of the test messages, such as whether the test messages were transmitted, received and/or resulted in a specified response, as well as the elapsed time for the test messages to be transmitted and/or received.

The message routing optimization system calculates a conversion rate for each routing provider based on the feedback data for the routing provider. The resulting conversion rate indicates the performance level of the routing provider, such as by indicating the likelihood that a message allocated to the routing provider will be successfully delivered to its intended recipient.

While selecting the routing provider with the highest conversion rate is key to providing optimized performance, the performance of the individual routing providers fluctuates over time and should be continually reassessed to maintain optimal performance. Accordingly, the message routing optimization system allocates messages to the routing providers with the mixed goals of achieving both exploitation of the optimal routing provider that is determined to provide the best performance based on the conversion rate index, and exploration of the secondary routing providers that are determined be providing lesser performance (e.g., lower than the optimal routing provider) based on the conversion rate index. Allocating messages for exploration purposes provides updated feedback data that is used to update the conversion rates and determine which routing provider is performing the best.

The message routing optimization system balances allocation of messages for exploration and exploitation purposes to provide optimized message routing performance. For example, the message routing optimization system attempts to maximize the number of messages allocated to the optimal routing provider, while also allocating enough messages to the secondary routing providers to allow for proper performance assessment of the routing providers.

To this end, the message routing optimization system allocates a portion of the messages to the optimal routing provider to exploit the higher performance provided by the optimal routing provider, and allocates another portion of the messages to the secondary routing providers to explore the performance provided by the secondary routing providers. For example, the message routing optimization system may allocate a certain percentage of the messages to each of the secondary routing providers for exploration purposes, with the remaining messages being allocated to the optimal routing provider. For example, the message routing optimization system may allocate 3% of the messages to each secondary routing provider for exploration purposes. Accordingly, in a case in which there are 5 total routing providers, 3% of the messages are allocated to each of the 4 secondary routing providers (e.g., 12% total), while the remaining 88% of the messages are allocated to the optimal routing provider.

To further encourage exploration, the message routing optimization system may allocate the messages to the routing providers based on an upper confidence bound estimate of the conversion rates for the routing providers. An upper confidence bound estimate provides an optimistic estimate of the performance of each routing provider, such as a most optimistic conversion rate for the routing provider based on the amount feedback data available. As the upper confidence bound estimate is based on available feedback data, the upper confidence bound estimate for routing providers with limited feedback data is amplified due to the limited amount feedback data available. For example, the upper confidence bound estimate for a routing provider with limited feedback may initially start very high, however will begin to fall as additional feedback data is gathered. Using the upper confidence bound estimate causes the message routing optimization system to allocate messages to routing providers with limited feedback data, resulting in better estimates in the performance of each routing provider.

