FIELD

Embodiments described herein relate to routing service requests, more specifically the embodiments relate to methods, systems, and policies for routing service requests to service instances in a service mesh.

BACKGROUND

Microservices are self-contained software components which may be built and deployed independently to perform specific functions. A single application may comprise or use a plurality of microservices which each separately perform individual functions of the application. Microservices are widely used to define modular cloud-native applications.

In use, copies, or instances, of microservices may be deployed on virtual processing and networking resources, such as within containers (within virtual machines) which are run on hardware resources such as servers. A single application may at any one time consist of a plurality of instances of a plurality of microservices, the microservice instances being deployed on different intercommunicating hardware resources within one or more networks, which in some cases may be a distributed cloud infrastructure. The deployment of microservices is performed by a container orchestration manager such as Kubernetes®.

The plurality of microservices of an application communicate with each other using a service mesh, which is a dedicated infrastructure layer for handling service-to-service communication. Service meshes operate by providing an array of service proxies, each of which is associated with a single microservice, or with a group, or pod, of multiple such microservices. Communications between the microservices are transmitted via the proxies and a communication network defined therebetween.

Service meshes consist of a data plane and a control plane. The data plane comprises the service proxies, the microservices and their communications, all of which are intercepted and transmitted via the service proxies. The control plane distributes policies and monitors the data plane.

Policies are distributed by the control plane to the service proxies of the data plane and control proxy behaviour without directly interacting with the service-to-service communications. Examples of policies include network management or security policies.

Policies distributed by the control plane are implemented by the service proxies and determine how service-to-service traffic is routed within the data plane, thereby allowing dynamic routing of service requests; for example, based on the type of request or to facilitate load balancing between the distributed and networked resources.

In addition to distributing policies to the service proxies, the control plane monitors the data plane and collects performance metrics thereon. Control planes may also perform other functions in service mesh, such as respond to queries from their associated data plane or assigning certificates to microservices.

Examples of widely used service meshes include Istio® and Linkerd®.

Where the microservices of an application are distributed across multiple hardware resources in a network, some or all of the communications between microservices must be transmitted between different hardware resources via the network.

Service meshes have historically been supported by high reliability, low latency networks. However, in some circumstances it may be advantageous to implement service meshes on dynamic networks which may experience changes in network performance. For example, a service mesh may be hosted on networks comprising both cloud resources, which are dynamically scalable in a cost effective manner, and local edge resources communicating via local network connections which may offer higher bandwidth and lower latency performance.

Changes in network performance are common on local wireless networks and wide area network connections will have limited capacities. Service-to-service communications in a service mesh hosted on such hybrid cloud edge networks may therefore experience changes in network performance.

In known cloud based service mesh solutions, performance statistics have been measured and used to determine how service requests should be routed. For example, wherein the control plane of a service mesh monitors the overall latency of service requests to specific microservice instances within the service mesh and implements routing policies to the proxies in response thereto. The overall latency of a service request to a microservice includes both the network or link latency of the communication to the microservice and the processing latency of the microservice itself, which are not distinguished in known monitoring solutions.

Such approaches consider the service mesh to be homogenous and to have high bandwidth network connections. The latency at a particular node of the service mesh has therefore been considered to be independent of the origin of service requests transmitted thereto and requests have only been routed and throttled in response to common shared information, such as historical service request latency statistics. In order to monitor the latency of individual links, it is helpful to know how virtual links between service proxies map onto physical links between the resources on which the microservices associated with the service proxies are hosted, which is not performed in known approaches.

Therefore, this approach has not accounted for variation in the performance of network links between hardware resources on which the microservices of the service mesh are run. Such performance variation occurs if nodes of the hosting are connected by wireless or lower bandwidth connections, which can cause performance bottlenecks.

An aim of the present embodiments described herein is to provide improvements to the routing of service requests in service meshes which account for variable network performance.

