We present our approach for deploying and managing distributed component-based applications. A Desired State Description (DSD), written in a high-level declarative language, specifies requirements for a distributed application. Our infrastructure accepts a DSD as input, and from it automatically configures and deploys the distributed application. Subsequent violations of the original requirements are detected and, where possible, automatically rectified by reconfiguration and redeployment of the necessary application components. A constraint solving tool is used to plan deployments that meet the application requirements.
The system described in this paper is most like @cite_18 which is an environment for a adapting distributed applications to heterogeneous environments. The framework relies on declarative specifications (like Deladas), a runtime environment called and an AI planner (described below). The specifications contain descriptions of components that specify the required and implemented interfaces and conditions that must hold for component instances. The runtime provides three distinct pieces of functionality: a proxy mechanism, wrapper components on each host and cache coherence mechanisms. The proxy mechanism causes requests for services to be sent to the AI planner which decides on the appropriate selection and placement of components. The wrapper abstracts over node specifics allowing (Java) components to be loaded onto nodes in a generic manner. The cache coherency mechanisms provide directory level cache coherence amongst nodes sharing replicated data.
@cite_18
2165204405 2014594876 2023682886 2131355481
Several recently proposed infrastructures permit client applications to interact with distributed network-accessible services by simply "plugging in" into a substrate that provides essential functionality, such as naming, discovery, and multi-protocol binding. However much work remains before the interaction can be considered truly seamless in the sense of adapting to the characteristics of the heterogeneous environments in which clients and services operate. This paper describes a novel approach for addressing this shortcoming: the partitionable services framework, which enables services to be flexibly assembled from multiple components, and facilitates transparent migration and replication of these components at locations closer to the client while still appearing as a single monolithic service. The framework consists of three pieces: (1) declarative specification of services in terms of constituent components; (2) run-time support for dynamic component deployment; and (3) planning policies, which steer the deployment to accomodate underlying environment characteristics. We demonstrate the salient features of the framework and highlight its usability and performance benefits with a case study involving a security-sensitive mail service. The ability to produce malleable parallel applications that can be stopped and reconfigured during the execution can offer attractive benefits for both the system and the applications. The reconfiguration can be in terms of varying the parallelism for the applications, changing the data distributions during the executions or dynamically changing the software components involved in the application execution. In distributed and Grid computing systems, migration and reconfiguration of such malleable applications across distributed heterogeneous sites which do not share common file systems provides flexibility for scheduling and resource management in such distributed environments. The present reconfiguration systems do not support migration of parallel applications to distributed locations. In this paper, we discuss a framework for developing malleable and migratable MPI message-passing parallel applications for distributed systems. The framework includes a user-level checkpointing library called SRS and a runtime support system that manages the checkpointed data for distribution to distributed locations. Our experiments and results indicate that the parallel applications, with instrumentation to SRS library, were able to achieve reconfigurability incurring about 15-35 overhead. The complexity of current and emerging architectures provides users with options about how best to use the available resources, but makes predicting performance challenging. In this work a benchmark-driven model is developed for a simple shallow water code on a Cray XE6 system, to explore how deployment choices such as domain decomposition and core affinity affect performance. The resource sharing present in modern multi-core architectures adds various levels of heterogeneity to the system. Shared resources often includes cache, memory, network controllers and in some cases floating point units (as in the AMD Bulldozer), which mean that the access time depends on the mapping of application tasks, and the core's location within the system. Heterogeneity further increases with the use of hardware-accelerators such as GPUs and the Intel Xeon Phi, where many specialist cores are attached to general-purpose cores. This trend for shared resources and non-uniform cores is expected to continue into the exascale era. The complexity of these systems means that various runtime scenarios are possible, and it has been found that under-populating nodes, altering the domain decomposition and non-standard task to core mappings can dramatically alter performance. To find this out, however, is often a process of trial and error. To better inform this process, a performance model was developed for a simple regular grid-based kernel code, shallow. The code comprises two distinct types of work, loop-based array updates and nearest-neighbour halo-exchanges. Separate performance models were developed for each part, both based on a similar methodology. Application specific benchmarks were run to measure performance for different problem sizes under different execution scenarios. These results were then fed into a performance model that derives resource usage for a given deployment scenario, with interpolation between results as necessary. Management, allocation and scheduling of heterogeneous resources for complex distributed real-time applications is a challenging problem. Timing constraints of applications may be fulfilled by the proper use of real-time scheduling policies, admission control and enforcement of timing constraints. However, it is not easy to design basic infrastructure services that allow for easy access to the allocation of multiple heterogeneous resources in a distributed environment. In this paper, we present a middleware for providing distributed soft real-time applications with a uniform API for reserving heterogeneous resources with real-time scheduling capabilities in a distributed environment. The architecture relies on standard POSIX OS facilities, such as time management and standard TCP IP networking services, and it is designed around CORBA, in order to facilitate modularity, flexibility and portability of the applications using it. However, real-time scheduling is supported by proper extensions at the kernel-level, plugged within the framework by means of dedicated resource managers. Our current implementation on Linux supports the reservation of the CPU, disk and network bandwidth. However, additional resource managers supporting alternative real-time schedulers for these resources, as well as additional types of resources, may be easily added. We present experimental results gathered on both synthetic applications and a real multimedia video streaming case study, showing the advantages deriving from the use of the proposed middleware. Finally, overhead figures are reported, showing the sustainability of the approach for a wide class of complex, distributed, soft real-time applications.