The Internet of Things (IoT) is offering unprecedented observational data that are used for managing Smart City utilities. Edge and Fog gateway devices are an integral part of IoT deployments to acquire real-time data and enact controls. Recently, Edge-computing is emerging as first-class paradigm to complement Cloud-centric analytics. But a key limitation is the lack of a platform-as-a-service for applications spanning Edge and Cloud. Here, we propose ( ECHO ), an orchestration platform for dataflows across distributed resources. ( ECHO ) ’s hybrid dataflow composition can operate on diverse data models – streams, micro-batches and files, and interface with native runtime engines like TensorFlow and Storm to execute them. It manages the application’s lifecycle, including container-based deployment and a registry for state management. ( ECHO ) can schedule the dataflow on different Edge, Fog and Cloud resources, and also perform dynamic task migration between resources. We validate the ( ECHO ) platform for executing video analytics and sensor streams for Smart Traffic and Smart Utility applications on Raspberry Pi, NVidia TX1, ARM64 and Azure Cloud VM resources, and present our results.
like Seti@home @cite_16 have targeted the use of idle compute capacity in desktops. However, some of the inherent P2P characteristics are missing in an IoT scenario. Device churn is a major factor in P2P but less so for infrastructure IoT, or even mobile devices that are typically within cell communication. This, coupled with the growth of global Cloud data centers, make it feasible for centralized services for coordination. Dataflow composition is also a non-goal for such P2P systems that typically use a task-queue model for opportunistic computing.
@cite_16
2008170189
Parallel computing on volatile distributed resources requires schedulers that consider job and resource characteristics. We study unconventional computing environments containing devices spread throughout a single large organization. The devices are not necessarily typical general purpose machines; instead, they could be processors dedicated to special purpose tasks (for example printing and document processing), but capable of being leveraged for distributed computations. Harvesting their idle cycles can simultaneously help resources cooperate to perform their primary task and enable additional functionality and services. A new burstiness metric characterizes the volatility of the high-priority native tasks. A burstiness-aware scheduling heuristic opportunistically introduces grid jobs (a lower priority workload class) to avoid the higher-priority native applications, and effectively harvests idle cycles. Simulations based on real workload traces indicate that this approach improves makespan by an average of 18.3 over random scheduling, and comes within 7.6 of the theoretical upper bound.