We consider a variant of the multi-armed bandit model, which we call multi-armed bandit problem with known trend, where the gambler knows the shape of the reward function of each arm but not its distribution. This new problem is motivated by different online problems like active learning, music and interface recommendation applications, where when an arm is sampled by the model the received reward change according to a known trend. By adapting the standard multi-armed bandit algorithm UCB1 to take advantage of this setting, we propose the new algorithm named A-UCB that assumes a stochastic model. We provide upper bounds of the regret which compare favourably with the ones of UCB1. We also confirm that experimentally with different simulations
Similar to @cite_5 , authors in @cite_3 propose a Thompson Sampling strategy equipped with a Bayesian change point mechanism to tackle this problem. They develop algorithms for a variety of cases with constant switching rate: when switching occurs all arms change (Global Switching), switching occurs independently for each arm (Per-Arm Switching), when the switching rate is known and when it must be inferred from data.
@cite_5 @cite_3
1944301273 2116082319
Thompson Sampling has recently been shown to achieve the lower bound on regret in the Bernoulli Multi-Armed Bandit setting. This bandit problem assumes stationary distributions for the rewards. It is often unrealistic to model the real world as a stationary distribution. In this paper we derive and evaluate algorithms using Thompson Sampling for a Switching Multi-Armed Bandit Problem. We propose a Thompson Sampling strategy equipped with a Bayesian change point mechanism to tackle this problem. We develop algorithms for a variety of cases with constant switching rate: when switching occurs all arms change (Global Switching), switching occurs independently for each arm (PerArm Switching), when the switching rate is known and when it must be inferred from data. This leads to a family of algorithms we collectively term Change-Point Thompson Sampling (CTS). We show empirical results in 4 articial environments, and 2 derived from real world data: news click-through[Yahoo!, 2011] and foreign exchange data[Dukascopy, 2012], comparing them to some other bandit algorithms. In real world data CTS is the most eective. The apparatus includes an arm pivoted between its end portions, a foot pedal for operating one of the end portions of the arm, and a switch operated by the other end portion of the arm. The pedal may be the accelerator pedal of an automotive vehicle. When the pedal is depressed, the switch operates immediately, even before the engine of the vehicle speeds up. The preferred switch has a special construction which is also part of the invention. In this paper, a physically interactive control scheme is developed for a manipulator robot arm. The human touches the robot and applies force in order to make it behave as he she likes. The communication between the robot and the human is maintained by a physical contact with no sensors. The intent of the human is estimated by observing the change in control effort. The robot receives the estimated human intent and updates its position reference accordingly. The developed method uses the principle of conservation of zero momentum for position-controlled systems. A switching scheme is developed that goes between the modes of pure impedance control with a fixed-position reference and interactive control under human intent. The switching mechanism uses neither a physical switch nor a sensor; it observes the human intent and puts the robot into interactive mode, if there is any. When the human intent disappears, the robot goes into the pure-impedance-control mode, thus stabilizing in the left position. A portable alarm device with a battery or low voltage operated sound signal triggered by a magnetic reed switch which is closed to complete the circuit by a magnet attached to a movably mounted arm, the poles of the magnet being positioned perpendicular to the longitudinal dimension of the contact strips of the reed switch to cause the reed switch to close when the magnet is in either of two positions relative to the switch thereby permitting the device to be used with doors which open inwardly or outwardly, with sliding doors, in drawers or with windows on two sides of a room. The device may also be constructed to activate a light signal simultaneously with the sound signal. In a modified form of the device, for use in the purse or in a pocket, the magnet is positioned with its poles parallel to the longitudinal dimension of the contact strips of the reed switch to cause the reed switch to close when the magnet is in one position over the center portion of the reed switch. Most datacenters still use Equal Cost Multi-Path (ECMP), which performs congestion-oblivious hashing of flows over multiple paths, leading to an uneven distribution of traffic. Alternatives to ECMP come with deployment challenges, as they require either changing the tenant VM network stacks (e.g., MPTCP) or replacing all of the switches (e.g., CONGA). We argue that the hypervisor provides a unique point for implementing load-balancing algorithms that are easy to deploy, while still reacting quickly to congestion. We propose Clove, a scalable load-balancer that (i) runs entirely in the hypervisor, requiring no modifications to tenant VM networking stacks or physical switches, and (ii) works on any topology and adapts quickly to topology changes and traffic shifts. Clove relies on standard ECMP in physical switches, discovers paths using a novel traceroute mechanism, uses software-based flowlet-switching, and continuously learns congestion (or path utilization) state using standard switch features. It then manipulates packet-header fields in the hypervisor switch to direct traffic over less congested paths. Clove achieves 1.5 to 7 times smaller flow-completion times at 70 network load than other load-balancing algorithms that work with existing hardware. Clove also captures some 80 of the performance gain of best-of-breed hardware-based load-balancing algorithms like CONGA that require new equipment. Pro-social punishment and exclusion are common means to elevate the level of cooperation among unrelated individuals. Indeed, it is worth pointing out that the combined use of these two strategies is quite common across human societies. However, it is still not known how a combined strategy where punishment and exclusion are switched can promote cooperation from the theoretical perspective. In this paper, we thus propose two different switching strategies, namely, peer switching that is based on peer punishment and peer exclusion, and pool switching that is based on pool punishment and pool exclusion. Individuals adopting the switching strategy will punish defectors when their numbers are below a threshold and exclude them otherwise. We study how the two switching strategies influence the evolutionary dynamics in the public goods game. We show that an intermediate value of the threshold leads to a stable coexistence of cooperators, defectors, and players adopting the switching strategy in a well-mixed population, and this regardless of whether the pool-based or the peer-based switching strategy is introduced. Moreover, we show that the pure exclusion strategy alone is able to evoke a limit cycle attractor in the evolutionary dynamics, such that cooperation can coexist with other strategies.Pro-social punishment and exclusion are common means to elevate the level of cooperation among unrelated individuals. Indeed, it is worth pointing out that the combined use of these two strategies is quite common across human societies. However, it is still not known how a combined strategy where punishment and exclusion are switched can promote cooperation from the theoretical perspective. In this paper, we thus propose two different switching strategies, namely, peer switching that is based on peer punishment and peer exclusion, and pool switching that is based on pool punishment and pool exclusion. Individuals adopting the switching strategy will punish defectors when their numbers are below a threshold and exclude them otherwise. We study how the two switching strategies influence the evolutionary dynamics in the public goods game. We show that an intermediate value of the threshold leads to a stable coexistence of cooperators, defectors, and players adopting the switching strategy in a well-mixed pop... We study a basic problem in Multi-Queue switches. A switch connects m input ports to a single output port. Each input port is equipped with an incoming FIFO queue with bounded capacity B. A switch serves its input queues by transmitting packets arriving at these queues, one packet per time unit. Since the arrival rate can be higher than the transmission rate and each queue has limited capacity, packet loss may occur as a result of insufficient queue space. The goal is to maximize the number of transmitted packets. This general scenario models most current networks (e.g., IP networks) which only support a “best effort” service in which all packet streams are treated equally. A 2-competitive algorithm for this problem was designed in [4] for arbitrary B. Recently, a ( 17 9 1.89 )-competitive algorithm was presented for B>1 in [2]. Our main result in this paper shows that for B which is not too small our algorithm can do better than 1.89, and approach a competitive ratio of ( e e-1 1.58 ). Closed kinematic chains are created whenever multiple robot arms concurrently manipulate a single object. The closed-chain constraint, when coupled with robot joint limits, dramatically changes the connectivity of the configuration space. We propose a regrasping move, termed “IK-switch,” which allows efficiently bridging components of the configuration space that are otherwise mutually disconnected. This move, combined with several other developments, such as a method to stabilize the manipulated object using the environment, a new tree structure, and a compliant control scheme, enables us to address complex closed-chain manipulation tasks, such as flipping a chair frame, which is otherwise impossible to realize using existing multi-arm planning methods. Hypervisors use software switches to steer packets to and from virtual machines (VMs). These switches frequently need upgrading and customization—to support new protocol headers or encapsulations for tunneling and overlays, to improve measurement and debugging features, and even to add middlebox-like functions. Software switches are typically based on a large body of code, including kernel code, and changing the switch is a formidable undertaking requiring domain mastery of network protocol design and developing, testing, and maintaining a large, complex codebase. Changing how a software switch forwards packets should not require intimate knowledge of its implementation. Instead, it should be possible to specify how packets are processed and forwarded in a high-level domain-specific language (DSL) such as P4, and compiled to run on a software switch. We present PISCES, a software switch derived from Open vSwitch (OVS), a hard-wired hypervisor switch, whose behavior is customized using P4. PISCES is not hard-wired to specific protocols; this independence makes it easy to add new features. We also show how the compiler can analyze the high-level specification to optimize forwarding performance. Our evaluation shows that PISCES performs comparably to OVS and that PISCES programs are about 40 times shorter than equivalent changes to OVS source code. The ever-increasing bandwidth requirements of modern datacenters have led researchers to propose networks based upon optical circuit switches, but these proposals face significant deployment challenges. In particular, previous proposals dynamically configure circuit switches in response to changes in workload, requiring network-wide demand estimation, centralized circuit assignment, and tight time synchronization between various network elements--- resulting in a complex and unwieldy control plane. Moreover, limitations in the technologies underlying the individual circuit switches restrict both the rate at which they can be reconfigured and the scale of the network that can be constructed. We propose RotorNet, a circuit-based network design that addresses these two challenges. While RotorNet dynamically reconfigures its constituent circuit switches, it decouples switch configuration from traffic patterns, obviating the need for demand collection and admitting a fully decentralized control plane. At the physical layer, RotorNet relaxes the requirements on the underlying circuit switches---in particular by not requiring individual switches to implement a full crossbar---enabling them to scale to 1000s of ports. We show that RotorNet outperforms comparably priced Fat Tree topologies under a variety of workload conditions, including traces taken from two commercial datacenters. We also demonstrate a small-scale RotorNet operating in practice on an eight-node testbed. A range of new datacenter switch designs combine wireless or optical circuit technologies with electrical packet switching to deliver higher performance at lower cost than traditional packet-switched networks. These "hybrid" networks schedule large traffic demands via a high-rate circuits and remaining traffic with a lower-rate, traditional packet-switches. Achieving high utilization requires an efficient scheduling algorithm that can compute proper circuit configurations and balance traffic across the switches. Recent proposals, however, provide no such algorithm and rely on an omniscient oracle to compute optimal switch configurations. Finding the right balance of circuit and packet switch use is difficult: circuits must be reconfigured to serve different demands, incurring non-trivial switching delay, while the packet switch is bandwidth constrained. Adapting existing crossbar scheduling algorithms proves challenging with these constraints. In this paper, we formalize the hybrid switching problem, explore the design space of scheduling algorithms, and provide insight on using such algorithms in practice. We propose a heuristic-based algorithm, Solstice that provides a 2.9× increase in circuit utilization over traditional scheduling algorithms, while being within 14 of optimal, at scale. Current technology trends make it possible to build communication networks that can support high-performance distributed computing. This paper describes issues in the design of a prototype switch for an arbitrary topology point-to-point network with link speeds of up to 1 Gbit s. The switch deals in fixed-length ATM-style cells, which it can process at a rate of 37 million cells per second. It provides high bandwidth and low latency for datagram traffic. In addition, it supports real-time traffic by providing bandwidth reservations with guaranteed latency bounds. The key to the switch's operation is a technique called parallel iterative matching , which can quickly identify a set of conflict-free cells for transmission in a time slot. Bandwidth reservations are accommodated in the switch by building a fixed schedule for transporting cells from reserved flows across the switch; parallel iterative matching can fill unused slots with datagram traffic. Finally, we note that parallel iterative matching may not allocate bandwidth fairly among flows of datagram traffic. We describe a technique called statistical matching , which can be used to ensure fairness at the switch and to support applications with rapidly changing needs for guaranteed bandwidth. The problem of maximizing the weighted throughput in various switching settings has been intensively studied recently through competitive analysis. To date, the most general model that has been investigated is the standard CIOQ (Combined Input and Output Queued) switch architecture with internal fabric speedup S ≥ 1. CIOQ switches, that comprise the backbone of packet routing networks, are N × N switches controlled by a switching policy that incorporates two components: Admission control and scheduling. An admission control strategy is essential to determine the packets stored in the FIFO queues in input and output ports, while the scheduling policy conducts the transfer of packets through the internal fabric, from input ports to output ports. The online problem of maximizing the total weighted throughput of CIOQ switches was recently investigated by Kesselman and Rosen [2003]. They presented two different online algorithms for the general problem that achieve non-constant competitive ratios (linear in either the speedup or the number of distinct values, or logarithmic in the value range). We introduce the first constant-competitive algorithm for the general case of the problem, with arbitrary speedup and packet values. Specifically, our algorithm is 8-competitive, and is also simple and easy to implement. The problem of efficiently sampling from a set of (undirected) graphs with a given degree sequence has many applications. One approach to this problem uses a simple Markov chain, which we call the switch chain, to perform the sampling. The switch chain is known to be rapidly mixing for regular degree sequences. We prove that the switch chain is rapidly mixing for any degree sequence with minimum degree at least 1 and with maximum degree dmax which satisfies 3 ≤ dmax ≤ 1 4 [EQUATION]M, where M is the sum of the degrees. The mixing time bound obtained is only an order of n larger than that established in the regular case, where n is the number of vertices. With high throughput networks acquiring a crucial role in supporting data-intensive applications, a variety of data center network topologies have been proposed to achieve high capacity at low cost. While this work explores a large number of design points, even in the limited case of a network of identical switches, no proposal has been able to claim any notion of optimality. The case of heterogeneous networks, incorporating multiple line-speeds and port-counts as data centers grow over time, introduces even greater complexity. In this paper, we present the first non-trivial upper-bound on network throughput under uniform traffic patterns for any topology with identical switches. We then show that random graphs achieve throughput surprisingly close to this bound, within a few percent at the scale of a few thousand servers. Apart from demonstrating that homogeneous topology design may be reaching its limits, this result also motivates our use of random graphs as building blocks for design of heterogeneous networks. Given a heterogeneous pool of network switches, we explore through experiments and analysis, how the distribution of servers across switches and the interconnection of switches affect network throughput. We apply these insights to a real-world heterogeneous data center topology, VL2, demonstrating as much as 43 higher throughput with the same equipment. A switched system consists of a finite number of subsystems and a rule that orchestrates switching among them. We develop notions of opacity for discrete-time switched linear systems. We distinguish between cases when the secret is specified as a set of initial modes, a set of initial states, or a combination of the two. The novelty of our schemes is in the fact that we place restrictions on: i) the allowed transitions between modes (specified by a directed graph), ii) the number of allowed changes of modes (specified by lengths of paths in the directed graph), and iii) the dwell times in each mode. Each notion of opacity is characterized in terms of allowed switching sequences and sets of reachable states and or modes. Finally, we present algorithmic procedures to verify these notions, and provide bounds on their computational complexity. We consider the problem of Bayesian inference for continuous-time multi-stable stochastic systems which can change both their diffusion and drift parameters at discrete times. We propose exact inference and sampling methodologies for two specific cases where the discontinuous dynamics is given by a Poisson process and a two-state Markovian switch. We test the methodology on simulated data, and apply it to two real data sets in finance and systems biology. Our experimental results show that the approach leads to valid inferences and non-trivial insights. Link bundling is a way to increase routing scalability whenever a pair of label switching routers in MPLS are connected by multiple parallel links. However, link bundling can be inefficient as a label switched path (LSP) has to be associated with a particular link. In this paper, we show that the efficiency of link bundling can be significantly improved if traffic can be effectively distributed across the parallel links. We propose an IP switch architecture that is capable of distributing flows both inside the switch and among the parallel links based on operations that are relatively simple to implement. The switch requires no speedup, guarantees in-sequence packet delivery for a given flow, avoids complex coordination algorithms, and can achieve LSP throughput higher than the line rate. By means of simulation using IP traces, we investigate the performance of the proposed switch, and show that the switch achieves good load-balancing performance. We describe extensions to the basic architecture which allows for very large bundle size, handles incremental upgrade strategy, improves reliability, and accommodates nonIP traffic. I. INTRODUCTION 1. Basic Concepts II . STABILITY OF SWITCHED SYSTEMS 2. Stability under Arbitrary Switching 3. Stability under Constrained Switching III. SWITCHING CONTROL 4. Systems not Stabilizable by Continuous Feedback 5. Systems with Sensor or Actuator Constraints 6. Systems with Large Modeling Uncertainty IV. SUPPLEMENTARY MATERIAL A. Stability B. Lie Algebras Notes and References Bibliography Index We present a study which investigated how and why users of Mozilla Firefox use multiple tabs and windows during web browsing. The detailed web browsing usage of 21 participants was logged over a period of 13 to 21 days each, and was supplemented by qualitative data from diary entries and interviews. Through an examination of several measures of their tab usage, we show that our participants had a strong preference for the use of tabs rather than multiple windows. We report the reasons they cited for using tabs, and the advantages over multiple windows. We identify several common tab usage patterns which browsers could explicitly support. Finally, we look at how tab usage affects web page revisitation. Most of our participants switched tabs more often than they used the back button, making tab switching the second most important navigation mechanism in the browser, after link clicking.