With huge amounts of training data, deep learning has made great breakthroughs in many artificial intelligence (AI) applications. However, such large-scale data sets present computational challenges, requiring training to be distributed on a cluster equipped with accelerators like GPUs. With the fast increase of GPU computing power, the data communications among GPUs have become a potential bottleneck on the overall training performance. In this paper, we first propose a general directed acyclic graph (DAG) model to describe the distributed synchronous stochastic gradient descent (S-SGD) algorithm, which has been widely used in distributed deep learning frameworks. To understand the practical impact of data communications on training performance, we conduct extensive empirical studies on four state-of-the-art distributed deep learning frameworks (i.e., Caffe-MPI, CNTK, MXNet and TensorFlow) over multi-GPU and multi-node environments with different data communication techniques, including PCIe, NVLink, 10GbE, and InfiniBand. Through both analytical and experimental studies, we identify the potential bottlenecks and overheads that could be further optimized. At last, we make the data set of our experimental traces publicly available, which could be used to support simulation-based studies.
S-SGD requires the set of computing units (e.g., GPUs) to exchange data iteratively, which can be implemented by either parameter server (PS) based methods @cite_27 @cite_2 or decentralized methods. In PS-based methods, there is one or more PSes that store the global model. The PS aggregates parameters at each iteration, updates the model, and then pushes the updated model to each computing unit. Performance models have been built by S. @cite_30 to generalize the performance of the PS-based methods, which provides guidelines for better system scalability.
@cite_30 @cite_27 @cite_2
2757938527 2060393849 2339765813
Photometric stereo (PS) is an established technique for high-detail reconstruction of 3D geometry and appearance. To correct for surface integration errors, PS is often combined with multiview stereo (MVS). With dynamic objects, PS reconstruction also faces the problem of computing optical flow (OF) for image alignment under rapid changes in illumination. Current PS methods typically compute optical flow and MVS as independent stages, each one with its own limitations and errors introduced by early regularization. In contrast, scene flow methods estimate geometry and motion, but lack the fine detail from PS. This paper proposes photogeometric scene flow (PGSF) for high-quality dynamic 3D reconstruction. PGSF performs PS, OF, and MVS simultaneously. It is based on two key observations: (i) while image alignment improves PS, PS allows for surfaces to be relit to improve alignment, (ii) PS provides surface gradients that render the smoothness term in MVS unnecessary, leading to truly data-driven, continuous depth estimates. This synergy is demonstrated in the quality of the resulting RGB appearance, 3D geometry, and 3D motion. There has been an increased growth in a number of applications that naturally generate large volumes of uncertain data. By the advent of such applications, the support of advanced analysis queries such as the skyline and its variant operators for big uncertain data has become important. In this paper, we propose the effective parallel algorithms using MapReduce to process the probabilistic skyline queries for uncertain data modeled by both discrete and continuous models. We present three filtering methods to identify probabilistic non-skyline objects in advance. We next develop a single MapReduce phase algorithm PS-QP-MR by utilizing space partitioning based on a variant of quadtrees to distribute the instances of objects effectively and the enhanced algorithm PS-QPF-MR by applying the three filtering methods additionally. We also propose the workload balancing technique to balance the workload of reduce functions based on the number of machines available. Finally, we present the brute-force algorithms PS-BR-MR and PS-BRF-MR with partitioning randomly and applying the filtering methods. In our experiments, we demonstrate the efficiency and scalability of PS-QPF-MR compared to the other algorithms. Join the Shortest Queue (JSQ) is a popular routing policy for server farms. However, until now all analysis of JSQ has been limited to First-Come-First-Serve (FCFS) server farms, whereas it is known that web server farms are better modeled as Processor Sharing (PS) server farms. We provide the first approximate analysis of JSQ in the PS server farm model for general job-size distributions, obtaining the distribution of queue length at each queue. To do this, we approximate the queue length of each queue in the server farm by a one-dimensional Markov chain, in a novel fashion. We also discover some interesting insensitivity properties of PS server farms with JSQ routing, and discuss the near-optimality of JSQ. This paper addresses the problem of photometric stereo for non-Lambertian surfaces. Existing approaches often adopt simplified reflectance models to make the problem more tractable, but this greatly hinders their applications on real-world objects. In this paper, we propose a deep fully convolutional network, called PS-FCN, that takes an arbitrary number of images of a static object captured under different light directions with a fixed camera as input, and predicts a normal map of the object in a fast feed-forward pass. Unlike the recently proposed learning based method, PS-FCN does not require a pre-defined set of light directions during training and testing, and can handle multiple images and light directions in an order-agnostic manner. Although we train PS-FCN on synthetic data, it can generalize well on real datasets. We further show that PS-FCN can be easily extended to handle the problem of uncalibrated photometric stereo. Extensive experiments on public real datasets show that PS-FCN outperforms existing approaches in calibrated photometric stereo, and promising results are achieved in uncalibrated scenario, clearly demonstrating its effectiveness. The problem of searching for mobile intruders in a polygonal region by mobile searchers is considered. A searcher can move continuously inside a polygon holding a flashlight that emits a single ray of light whose direction can be changed continuously. The vision of a searcher at any time instant is limited to the points on the ray. The intruders can move continuously with unbounded speed. We denote by ps(P) the polygon search number of a simple polygon P , which is the number of searchers necessary and sufficient to search P . Let n , r , b , and g be the number of edges, the number of reflex vertices, the bushiness, and the size of a minimum guard set of P , respectively. In this paper we present matching upper and (worst case) lower bounds of 1 + log 3 (2b+1) on ps(P) . Also upper bounds on ps(P) in terms of n,r , and g are presented;ps(P) ≤ 1 + log 3 (n-3) ,ps(P) ≤ 1 + log 3 r , and ps(P) ≤ 2 + log 2 g . These upper bounds are tight or almost tight in the worst case, since we show that for any natural number s 2 , there is a polygon P such that ps(P) = log 3 (n+1) = log 3 (2r+3) = 1 + log 3 (2g-1) = s . We study the problem of polynomial identity testing (PIT) for depth 2 arithmetic circuits over matrix algebra. We show that identity testing of depth 3 (SPS) arithmetic circuits over a field F is polynomial time equivalent to identity testing of depth 2 (PS) arithmetic circuits over U2(F), the algebra of upper-triangular 2 2 matrices with entries from F. Such a connection is a bit surprising since we also show that, as computational models, PS circuits over U2(F) are strictly ‘weaker’ than SPS circuits over F. The equivalence further implies that PIT of SPS circuits reduces to PIT of width-2 commutative Algebraic Branching Programs(ABP). Further, we give a deterministic polynomial time identity testing algorithm for a PS circuit of size s over commutative algebras of dimension O(log s log log s) over F. Over commutative algebras of dimension poly(s), we show that identity testing of PS circuits is at least as hard as that of SPS circuits over F. In a scheduling problem where agents can opt out, we show that the familiar random priority RP mechanism can be improved upon by another mechanism dubbed probabilistic serial PS. Both mechanisms are nonmanipulable in a strong sense, but the latter is Pareto superior to the former and serves a larger expected number of agents. The PS equilibrium outcome is easier to compute than the RP outcome; on the other hand, RP is easier to implement than PS. We show that the improvement of PS over RP is significant but small: at most a couple of percentage points in the relative welfare gain and the relative difference in quantity served. Both gains vanish when the number of agents is large; hence both mechanisms can be used as a proxy of each other. We consider a distributed server system in which heterogeneous servers operate under the processor sharing (PS) discipline. Exponentially distributed jobs arrive to a dispatcher, which assigns each task to one of the servers. In the so-called size-aware system, the dispatcher is assumed to know the remaining service requirements of some or all of the existing jobs in each server. The aim is to minimize the mean sojourn time, i.e., the mean response time. To this end, we first analyze an M M 1-PS queue in the framework of Markov decision processes, and derive the so-called size-aware relative value of state, which sums up the deviation from the average rate at which sojourn times are accumulated in the infinite time horizon. This task turns out to be non-trivial. The exact analysis yields an infinite system of first order differential equations, for which an explicit solution is derived. The relative values are then utilized to develop efficient dispatching policies by means of the first policy iteration (FPI). Numerically, we show that for the exponentially distributed job sizes the myopic approach, ignoring the future arrivals, yields an efficient and robust policy when compared to other heuristics. However, in the case of highly asymmetric service rates, an FPI based policy outperforms it. Additionally, the size-aware relative value of an M G 1-PS queue is shown to be sensitive with respect to the form of job size distribution, and indeed, the numerical experiments with constant job sizes confirm that the optimal decision depends on the job size distribution. Computer keyboards are often used to transmit confidential data such as passwords. Since they contain electronic components, keyboards eventually emit electromagnetic waves. These emanations could reveal sensitive information such as keystrokes. The technique generally used to detect compromising emanations is based on a wide-band receiver, tuned on a specific frequency. However, this method may not be optimal since a significant amount of information is lost during the signal acquisition. Our approach is to acquire the raw signal directly from the antenna and to process the entire captured electromagnetic spectrum. Thanks to this method, we detected four different kinds of compromising electromagnetic emanations generated by wired and wireless keyboards. These emissions lead to a full or a partial recovery of the keystrokes. We implemented these sidechannel attacks and our best practical attack fully recovered 95 of the keystrokes of a PS 2 keyboard at a distance up to 20 meters, even through walls. We tested 12 different keyboard models bought between 2001 and 2008 (PS 2, USB, wireless and laptop). They are all vulnerable to at least one of the four attacks. We conclude that most of modern computer keyboards generate compromising emanations (mainly because of the manufacturer cost pressures in the design). Hence, they are not safe to transmit confidential information. Communication is an enabling technology for the efficient control and management of next-generation Smart Grids. Energy conservation of the communication devices is essential for future large scale deployment of Smart Grid communication networks. However, existing power save protocols experience high contention in Smart Grid communication networks that have a large number of nodes and periodic traffic. We design a new energy conservation protocol, Power Save with Offset Listen Interval (PS-OLi), to address such contention problems. PS-OLi avoids message collisions by controlling the station wake up time with a calculated offset. A new analytical model is developed to characterize the power save performance of networks with periodic traffic. Simulation results show that our analytical model accurately predicts the collision probability and packet delay. We use our model to evaluate the energy efficiency of PS-OLi and standard power save protocols. Our results show that PS-OLi extends the lifetime of a Smart Grid communication network by more than 10 . In this paper we present the first solution to 3D reconstruction in monocular laparoscopy using methods based on Photometric Stereo (PS). Our main contributions are to provide the new theory and practical solutions to successfully apply PS in close-range imaging conditions. We are specifically motivated by a solution with minimal hardware modification to existing laparoscopes. In fact the only physical modification we make is to adjust the colour of the laparoscope's illumination via three colour filters placed at its tip. Once calibrated, our approach can compute 3D from a single image, does not require correspondence estimation, and computes absolute depth densely. We demonstrate the potential of our approach with ground truth ex-vivo and in-vivo experimentation. We propose a novel methodology for re-identification, based on Pictorial Structures (PS). Whenever face or other biometric information is missing, humans recognize an individual by selectively focusing on the body parts, looking for part-to-part correspondences. We want to take inspiration from this strategy in a re-identification context, using PS to achieve this objective. For single image re-identification, we adopt PS to localize the parts, extract and match their descriptors. When multiple images of a single individual are available, we propose a new algorithm to customize the fit of PS on that specific person, leading to what we call a Custom Pictorial Structure (CPS). CPS learns the appearance of an individual, improving the localization of its parts, thus obtaining more reliable visual characteristics for re-identification. It is based on the statistical learning of pixel attributes collected through spatio-temporal reasoning. The use of PS and CPS leads to state-of-the-art results on all the available public benchmarks, and opens a fresh new direction for research on re-identification. In this work, we maximize the secrecy rate of the wireless-powered untrusted relay network by jointly designing power splitting (PS) ratio and relay beamforming with the proposed global optimal algorithm (GOA) and local optimal algorithm (LOA). Different from the literature, artificial noise (AN) sent by the destination not only degrades the channel condition of the eavesdropper to improve the secrecy rate, but also becomes a new source of energy powering the untrusted relay based on PS. Hence, it is of high economic benefits and efficiency to take advantage of AN compared with the literature. Simulation results show that LOA can achieve satisfactory secrecy rate performance compared with that of GOA, but with less computation time. Public Safety (PS) organizations bring value to society by creating a stable and secure environment. The services they provide include protection of people, environment and assets and they address a large number of threats both natural and man-made, acts of terrorism, technological, radiological or environmental accidents. The capability to exchange information (e.g., voice and data) is essential to improve the coordination of PS officers during an emergency crisis and improve response efforts. Wireless communications are particularly important in field operations to support the mobility of first responders. Recent disasters have emphasized the need to enhance interoperability, capacity and broadband connectivity of the wireless networks used by PS organizations. This paper surveys the outstanding challenges in this area, the status of wireless communication technologies in this particular domain and the current regulatory, standardization and research activities to address the identified challenges, with a particular focus on USA and Europe. This paper introduces a novel approach to texture synthesis based on generative adversarial networks (GAN) (, 2014), and call this technique Periodic Spatial GAN (PS-GAN). The PSGAN has several novel abilities which surpass the current state of the art in texture synthesis. First, we can learn multiple textures, periodic or non-periodic, from datasets of one or more complex large images. Second, we show that the image generation with PS-GANs has properties of a texture manifold: we can smoothly interpolate between samples in the structured noise space and generate novel samples, which lie perceptually between the textures of the original dataset. We make multiple experiments which show that PSGANs can flexibly handle diverse texture and image data sources, and the method is highly scalable and can generate output images of arbitrary large size. This paper considers the transmitter design for wireless information and energy transfer (WIET) in a multiple-input single-output (MISO) interference channel (IFC). The design problem is to maximize the system throughput subject to individual energy harvesting constraints and power constraints. It is observed that the ideal scheme, where the receivers simultaneously perform information detection (ID) and energy harvesting (EH) from the received signal, may not always achieve the best tradeoff between information transfer and energy harvesting, but simple practical schemes based on time splitting may perform better. We therefore propose two practical time splitting schemes, namely the time-division mode switching (TDMS) and time-division multiple access (TDMA), in addition to the existing power splitting (PS) scheme. In the two-user scenario, we show that beamforming is optimal to all the schemes. Moreover, the design problems associated with the TDMS and TDMA schemes admit semi-analytical solutions. In the general K-user scenario, a successive convex approximation method is proposed to handle the WIET problems associated with the ideal scheme, the PS scheme and the TDMA scheme, which are known NP-hard in general. Simulation results show that none of the schemes under consideration can always dominate another in terms of the sum rate performance. Specifically, it is observed that stronger cross-link channel power improves the achievable sum rate of time splitting schemes but degrades the sum rate performance of the ideal scheme and PS scheme. As a result, time splitting schemes can outperform the ideal scheme and the PS scheme in interference dominated scenarios. The knowledge about the software metrics which serve as defect indicators is vital for the efficient allocation of resources for quality assurance. It is the process metrics, although sometimes difficult to collect, which have recently become popular with regard to defect prediction. However, in order to identify rightly the process metrics which are actually worth collecting, we need the evidence validating their ability to improve the product metric-based defect prediction models. This paper presents an empirical evaluation in which several process metrics were investigated in order to identify the ones which significantly improve the defect prediction models based on product metrics. Data from a wide range of software projects (both, industrial and open source) were collected. The predictions of the models that use only product metrics (simple models) were compared with the predictions of the models which used product metrics, as well as one of the process metrics under scrutiny (advanced models). To decide whether the improvements were significant or not, statistical tests were performed and effect sizes were calculated. The advanced defect prediction models trained on a data set containing product metrics and additionally Number of Distinct Committers (NDC) were significantly better than the simple models without NDC, while the effect size was medium and the probability of superiority (PS) of the advanced models over simple ones was high ( @math p = . 016 , @math r = - . 29 , @math PS = . 76 ), which is a substantial finding useful in defect prediction. A similar result with slightly smaller PS was achieved by the advanced models trained on a data set containing product metrics and additionally all of the investigated process metrics ( @math p = . 038 , @math r = - . 29 , @math PS = . 68 ). The advanced models trained on a data set containing product metrics and additionally Number of Modified Lines (NML) were significantly better than the simple models without NML, but the effect size was small ( @math p = . 038 , @math r = . 06 ). Hence, it is reasonable to recommend the NDC process metric in building the defect prediction models. We consider a distributed server system with m servers operating under the processor sharing (PS) discipline. A stream of fixed size tasks arrives to a dispatcher, which assigns each task to one of the servers. We are interested in minimizing the mean sojourn time, i.e., the mean response time. To this end, we first analyze an M D l-PS queue in the MDP framework. In particular, we derive a closed form expression for the so-called size-aware relative value of state, which sums up the deviation from the average rate at which sojourn times are accumulated in the infinite time horizon. This result can be applied in numerous situations. Here we give an example in the context of dispatching problems by deriving efficient and robust state-dependent dispatching policies for homogeneous and heterogeneous server systems. The obtained policies are further demonstrated by numerical examples. As a newly emerging network model, heterogeneous information networks (HINs) have received growing attention. Many data mining tasks have been explored in HINs, including clustering, classification, and similarity search. Similarity join is a fundamental operation required for many problems. It is attracting attention from various applications on network data, such as friend recommendation, link prediction, and online advertising. Although similarity join has been well studied in homogeneous networks, it has not yet been studied in heterogeneous networks. Especially, none of the existing research on similarity join takes different semantic meanings behind paths into consideration and almost all completely ignore the heterogeneity and diversity of the HINs. In this paper, we propose a path-based similarity join (PS-join) method to return the top @math similar pairs of objects based on any user specified join path in a heterogeneous information network. We study how to prune expensive similarity computation by introducing bucket pruning based locality sensitive hashing (BPLSH) indexing. Compared with existing Link-based Similarity join (LS-join) method, PS-join can derive various similarity semantics. Experimental results on real data sets show the efficiency and effectiveness of the proposed approach. This paper studies a multi-user multiple-input single-output (MISO) downlink system for simultaneous wireless information and power transfer (SWIPT), in which a set of single-antenna mobile stations (MSs) receive information and energy simultaneously via power splitting (PS) from the signal sent by a multi-antenna base station (BS). We aim to minimize the total transmission power at BS by jointly designing transmit beamforming vectors and receive PS ratios for all MSs under their given signal-to-interference-plus-noise ratio (SINR) constraints for information decoding and harvested power constraints for energy harvesting. First, we derive the sufficient and necessary condition for the feasibility of our formulated problem. Next, we solve this non-convex problem by applying the technique of semidefinite relaxation (SDR). We prove that SDR is indeed tight for our problem and thus achieves its global optimum. Finally, we propose two suboptimal solutions of lower complexity than the optimal solution based on the principle of separating the optimization of transmit beamforming and receive PS, where the zero-forcing (ZF) and the SINR-optimal based transmit beamforming schemes are applied, respectively.